100 days of YARA Challenge

Background

Recently, I completed the 100 days of YARA challenge which was originally started by Greg Lesnewich (Twitter Handle: @greglesnewich) and has been going on since 2022. I thought of sharing my experience and maybe motivate a few to take on the challenge next year. For those who do not know about the challenge, it is essentially that each day for a total of 100 days, you write and contribute a YARA rule that you have written. Feel free to check out more details about the challenge here.

Prior to this challenge, I had only heard about YARA and never wrote a single rule. To make things even more challenging, I only started the challenge 16 days later and had plenty of catch up to do (backlog of 16 rules). At this point, I was simultaneously trying to learn YARA and write a rule each day along with my work which was pretty difficult to begin with. Having said that, I would highly recommend everyone to give it a go. By the end of the challenge, there were only three people who successfully completed it.

In my opinion, as far as YARA goes or for that matter anything new that you want to learn, you can go through the entirety of the official documentation, watch videos on it, read blogs and what not. But if you do not start writing your own YARA rules, you will not get any better at it and this is exactly what the challenge enabled me to do. By pushing a commitment to write a rule each day, you are building an appetite for it. When I look back at the rules I started with versus the rules written towards the rest of the challenge, I could see myself improve bit by bit. From writing string based rules to using various modules in YARA and then transitioning towards writing rules based on malicious behaviour (TTPs).

This is not a guide to writing YARA from scratch (there is plenty of content available online talking about the syntax, examples and much more) but rather special considerations while writing YARA and hopefully pumping you up to get cracking with it.

Here is my advice to those starting fresh with YARA:

• Don’t wait to prepare yourself for the challenge: I jumped straight into it and had some solid ground to cover but ended up not only completing the challenge but also getting better at YARA and receiving kudos for the rules I wrote from the community. Throughout this journey, I did make some mistakes but thanks to the awesome people in our community, they helped me out massively through each step.

• You do not need to know Malware Stuff: There are no real prerequisites and many people have the misconception that better YARA rules are governed by having a malware analysis background, This is not entirely true, YARA is about finding patterns and as you read rules written by other people and start writing your own rules, you will eventually start to recognise such suspicious patterns. While understanding malware characteristics can be helpful to inform your YARA rules, by no means it is a show stopper. Sure malware analysis can assist in writing YARA based on OP codes, API hashing or stack strings but while getting started don’t worry about the fancy stuff.

• Not a silver bullet: YARA is meant for static analysis. Whatever rule you might come up with does not need to detect each and every malware variant for the malware family it is related to. Chances are slim to none that it will happen anyway. This was certainly my natural tendency to create really broad rules and try to match every sample in that malware family. This is next to impossible. A decent YARA rule is something that matches on more than a single malware sample and is stirring clear of false positives.

• Keep it simple stupid (KISS): You do not need to write YARA rules which use built in YARA modules. In fact pretty much anything that is available through the YARA modules can be done using strings alone. You can even treat the strings capability that YARA brings as a superset to what is possible with the modules with the difference being that modules are more structured and specific since they parse the characteristics more effectively.

• Be open to experimenting: Once you get a good grip on the YARA syntax and start getting comfortable with writing your own YARA rules independently, start experimenting with your rules and also play around with the different tools available. This will allow you to understand what you like and maybe address some of the challenges you might be facing (more on selection of tooling later). One of the mistakes I made during most of the challenge was to limit my scope of improvement to using additional modules and not thinking beyond that.

There are a bunch of guides available which can give a big boost to your YARA rules. I have shortlisted some of the ones that you can easily add to your rules to boost their quality.

I would highly recommend checking out Florian Roth’s Part 1 and Part 2 blogs for a more in-depth understanding on writing better YARA rules.

• Use file headers: This allows you to limit your rule to a specific file type (for instance windows executable or DLLs). This is very useful to reduce false positives from matching on any other file.

• Use File Size: YARA evaluates this condition after checking all other conditions first. This is a good practice to avoid matching on large samples as it can exhaust your computing resources and consume time especially while running YARA against a system with large file directories.

• Use full words: The Full Words identifier is very helpful for those strings which are smaller in length. This is because smaller words can be part of bigger strings which can result in false positives or FPs. What this identifier does is tell YARA to match on this string with this particular identifier only if the string matches completely. By default YARA will match strings just like how you would use the CTRL + F feature on any string found in the text.

Just by doing the above, you can improve the quality of your YARA rules

Considerations while writing rules

There are several factors that will influence the kind of preference you will have while writing your rules. To break down some of the main ones, have a look at these:

• All systems Go: Access to an isolated computer or a virtual machine or just online sources will influence the tools that you can use at your disposal to write YARA.

• Racing against time: Time will also be a contributing factor around writing YARA rules. The initial setup to run YARA locally on your own system is time consuming. First, you need to install YARA which is fine but then you also need a malware corpus to run your rules against. VX underground is the way to go, they offer yearly archives which come in different sizes ranging in gigabytes. It is also a good practice to have a good ware corpus to test against false positives (FPs) which will not only take time to setup but also consume storage.

• Sample Size: You will start to realise quite quickly that the quality of the YARA rules you write heavily rely on your malware corpus. A good way to measure the quality of your malware corpus is to look at your sample size and the corresponding dates of those malware samples. For instance, VX underground do yearly archives of their collected malware samples which is invaluable. I ended up grabbing their yearly collections from 2019 to 2023 before running out of storage on my virtual machine.

• Safety first: This is super important, if you do plan to run YARA locally which I very much am rooting for in order to get that premium experience, you got to make sure that you isolate your machine running YARA even though we are only testing these samples statically without detonating the malware samples. But mistakes can happen, it is easy to accidently double click on an executable! Do not be a rusty noob, be better.

Side note: I had to trouble downloading the samples from VX underground onto my windows machine because Windows does not like filenames containing non-ASCII characters which was quite annoying. I also eventually ran out of storage on my VM which had around 80 GB allocated to it. It might be worth considering deploying YARA on a linux system. This is also a safer approach since most of the malware would be for Windows. I have heard that it is more performance friendly (not sure) but more importantly might fix the issues I had while running YARA on Windows10.

Work Flow

There is this one meme that comes to my mind when thinking about the work flow and what would be considered a successful YARA rule (Meme credits @lauriewired ).

At a very high level, your YARA rule should be :

1. Write a YARA: Yup, this is a very obvious step.
2. Error Free: It should compile without any errors, you really do not want to be distributing YARA rules that won’t even run in the first place.
3. Match against more than one sample at least: If your YARA only matches against one file, you got to ask yourself, what is the difference between my rule and a file hash.
4. Test against Good stuff: It is one thing that your YARA matches against sus files, but does it also match against the good stuff (system files which are not malicious in nature). This is a common beginner mistake as you try to familiarize yourself with YARA and what bad looks like. For instance, in the beginning, I picked strings for my rule which were found in good ware or those which are generated during run time (these wont match again on other samples). By being aware that these things can happen, you can very effectively reduce FPs.

Write a Rule => Check for Errors => Test against Bad => Test against good

Tooling

While I will be talking about the tools that can assist you in writing YARA rules depending on what you have access to and the time constraints, it is crucial to understand that methodology is primary and these tools are secondary. Without having a game plan in place, you might not be able to benefit from the different tooling that are available. Focus on your approach and try to think of a methodology that you can build which aligns with the work flow.

I might write a simple blog highlighting what approach I had taken towards writing YARA and also share a high level methodology behind the approach taken

Old School

As I briefly mentioned above, I would highly recommend to install YARA locally. But this can be cumbersome. You need the time, patience and storage to accommodate the requirements that come with this setup. I am pretty old school when it comes to pretty much anything. I prefer holding a book rather than reading the same material online (even though I read slowly and the average number of books I have read in a year is less than 5). This setup really allows us to understand the true capability that YARA brings to the table. But it does put you in a tight spot when it comes to the initial setup and the time taken to complete it.

Pros:

• Learning by doing
• Access to all features
• Ability to perform additional data analysis to detect what is common and rare in your corpus

Cons:

• Initial setup is time consuming
• Running out of storage
• Manually updating your corpus over time

YARAify by Abuse Ch

Wouldn’t it be cool to see how your rule performs against the samples found in the wild beyond your malware corpus. This is definitely within your arms reach. This amazing service by the same guys who brought you malware bazaar is really handy to test your rules out. Having used malware bazaar to gain access to malware samples and using threat fox to grab indicators for C2 hunting, I have always been a huge fan of Abuse Ch and the fact that they are community driven is absolutely amazing.

Here, have a look at the stats of my YARA rules from the challenge, you can also check out my YARAify user profile here …

I am planning to release a separate blog on YARAify as I am absolutely in love with the platform and think that it will be a gold mine for beginners.

Pros:

• Testing your YARA rules without any samples
• Support for all YARA modules
• Large sample size (over 1 million files)
• Control over sharing the YARA rule with the community
• Free

Cons:

• No Retro Lookup
• Manual upload of YARA rule with required metadata

Hybrid Analysis

Similar capabilities to YARAify, this service allows you to scan your YARA rule instantly (retro lookup). However this comes with a drawback, no modules are supported using this service which makes the usage quite limited to string based rules only. You could treat this as a reinforcer that string based rules are still OP (Over Powered). In fact why not test your skills out by converting some of the module based rules into string based ones.

Pros:

• Testing your YARA rules online
• Instantly returns matched files
• Free of cost (requires registration)

Cons:

• No ongoing notification for matched files
• No modules supported
• Unknown sample size
• No control over TLP for sharing YARA with community

Klara by Kaspersky

This utility allows a hybrid setup where you install the python based tool on your system and not having to worry about malware compromising your machine. This is basically a querying service but installed on your system. I personally have not tested it yet, but looks very promising. Here is the link to their GitHub.

Pros:

• Test YARA rules against good and bad samples

Cons:

• Requires further testing to determine sample size, sample update frequency and overall efficacy while trying YARA

Virus Total Intelligence (Premium)

This is a very convenient alternative to most of the tooling mentioned so far only if you have this service deployed in your organisation. Virus Total has both the Retro hunt and Live hunt capabilities. Both have the same online editors to write your YARA rules in and also validate the syntax. In order to test your rules, all you need is the file hash. You can test up to 50 files in a single scan which is plenty. In order to test the syntax and primary matching against known bad as well as good, this is a really solid option.

They also have a good ware corpus and running YARA against it does not cost credits neither does testing your rule in their editor. VT has the capability to fetch the strings (ASCII and Unicode) from the sample along with a Hex view. VT also has a custom module “vt” which has opens up YARA to a whole lot of other stuff such as dynamic analysis, network based characteristics, pivoting between different file types and much more.

Even beyond VT intelligence, Virus Total can be used t conduct initial recon against files such as looking at their PE characteristics (provided they are a windows exe or Dll). This is a useful feature which eliminates the need to run the PE module against a sample locally and the best part is that this feature does not require for users to log into VT.

These are some of the VT based YARA rules that I wrote during the challenge:

• Rule to track APT Transparent Tribe infrastructure

• Rule to detect Remcos RAT based on the JA3 fingerprint

• Rule to discover HookBot C2 Login Panels based on HTML properties

I might cover the capabilities of VT Intel for YARA in the future as it has a customr VT module which opens YARA to a lot of interesting use cases, let me know if this might be useful.

Pros:

• Everything in one place
• Testing against Goodware corpus for free provided access with purchased
• Provided access is enabled, the core use cases mentioned in the workflow is free of cost

Cons:

• Paid Subscription required
• Custom VT module cannot be used else where
• Retro scans and live hunts cost credits

As you can see from the above, each tool brings something different to the table and ideally it would be a fantastic idea to run all the above tooling in order to cover maximum ground and capitalise on the various limitations. I would like to propose a more practical setup and this goes back to the work flow that I briefly mentioned.

1. Error Free: Grab any text editor that you prefer to write YARA on. For me, the winner is VS Studio Code since it has a neat YARA extension that you can install which makes syntax editing easier which is very useful to debug any errors in your YARA rules.

2. Double Tap: The YARA toolkit by @Frogger so very useful to write your YARA rule online and also validate it (more on this later). It also provides small snippets of YARA rules which are very easy to look at and maybe even get some ideas for your own rules.

3. Test against Bad: Ideally this would be on your own system running YARA to validate that at a bare minimum your rule matches with the sample you used to create your rule in the first place. If you do not have this, then your next best bet would be YARAify. It also enables you to upload the malware sample or a Zip file (packed) containing your sample to the platform and then run your YARA rule against it.

4. Test against Good: The easiest way to accomplish this would be to simply install YARA on your own machine and run it against a specified file directory. This is perfectly safe and if your rule triggers on your own system then you better have the incident response on your speed dial.

5. Optional but Fun: Upload your YARA rule to YARAify to monitor the file matches against your rules over a period of time. Do be careful as this can be super addictive, I nearly ended up staying up the whole night just watching the positive hits on my YARA rules.

Other Sources:

Finally to end this blog, I would like to share some really helpful resources which made writing YARA even easier

YARA official docs: These official docs will be your best friend while you get familiarised with the syntax and the modules that YARA has to offer. I found myself referencing these through out the challenge particularly for the PE module.

YARA Toolkit This is an absolute gold mine of a resource created by Thomas Roccia @fr0gger_ . The YARA Toolkit contains some cool utilities which can help speed up the process of creating YARA rules as well as carry out more complex tasks, all from the online tool. Bonus: it also contains a collection of small snippets of YARA rules which can be added to your rules such as the PE file header check YARA.

Malapi: The last resource is the MalAPI.io which was created by Mr. Dox @mrd0x and was originally intended for malware analysts and reverse engineers. Understanding import functions and the libraries that malware authors use can be challenging and this is the problem that this resource attempts to solve by mapping the suspicious API functions to the tactics that the threat actors are attempting to complete.

Bonus YARA Rule

This is all for now, please feel free to reach out to me on Twitter @RustyNoob619 and let me know what topic would you like my next blog to cover. I will be dropping a poll on Twitter soon to see what I should write next, until then, stay frosty…