Antiemulation Techniques (Malware Tricks II)

2010, Feb 23    

From time to time, when reversing malware, I find new antiemulation techniques as they are widely used by malware to evade detection by AVs that uses emulation, however, it seems that no one wrote about them maybe because there are a lot or, maybe, because they aren’t very interesting. Anyway, a friend and I decided to look for antiemulation techniques and we found a bunch of them in just about 2 days. Surprise. Well, the following is a list of antiemulation techniques “found” by us.

API Emulation

The most typically used antiemulation technique is the use of undocumented APIs or the use of non common ones such as, in example, SetErrorMode:

DWORD dwCode = 1024;
  1.  
  2.   SetErrorMode(1024);
  3.   if (SetErrorMode() != 1024)
  4.     printf("Hi emulator!\n");

This technique catches, at least, the IDAPro+Bochs debugger and Norman Sandbox.

Another typical trick is the use of non existent APIs. Many emulators will try to “emulate” the function by simply returning 0 instead of failing with a null pointer exception. Another one, try to load a vital library for the operating system which is not emulated and call an exported function: just trying to load the library will fail in almost any emulators:

int test6(void)
  1. {
  2. HANDLE hProc;
  3.  
  4.     hProc = LoadLibrary("ntoskrnl.exe");
  5.  
  6.     if (hProc == NULL)
  7.         return EMULATOR_DETECTED;
  8.     else
  9.         return EMULATOR_NOT_DETECTED;
  10. }

Just in the case an emulator allows to load any library returning a pseudo handle, a bit more complex examples:

struct data1
  1. {
  2.   int a1;
  3.   int a2;
  4. };
  5.  
  6. struct data2
  7. {
  8.   int a1;
  9.   int a2;
  10.   int a3;
  11.   int a4;
  12.   int a5;
  13.   int a6;
  14.   struct data1 *a7;
  15. };
  16.  
  17. typedef int (WINAPI *FCcSetReadAheadGranularity)(struct data2 *a1, int num);
  18. typedef int (WINAPI *FIofCallDriver)();
  19.  
  20. int test8(void)
  21. {
  22. HINSTANCE hProc;
  23. FIofCallDriver pIofCallDriver;
  24.  
  25.  hProc = LoadLibrary("ntkrnlpa.exe");
  26.  
  27.  if (hProc == NULL)
  28.   return ;
  29.  
  30.  pIofCallDriver = (FIofCallDriver) GetProcAddress(hProc, "IofCallDriver");
  31.  pIofCallDriver -= 2; // At this point there is a 0xCC character, so an INT3 should be raised
  32.  
  33.  try
  34.  {
  35.   pIofCallDriver();
  36.   return EMULATOR_DETECTED;
  37.  }
  38.  catch(…)
  39.  {
  40.   return EMULATOR_NOT_DETECTED;
  41.  }
  42.  
  43. }
  44.  
  45. int test9(void)
  46. {
  47. HINSTANCE hProc;
  48. FCcSetReadAheadGranularity CcSetReadAheadGranularity;
  49. struct data1 s1;
  50. struct data2 s2;
  51. int ret;
  52.  
  53.  hProc = LoadLibrary("ntkrnlpa.exe");
  54.  
  55.  if (hProc == NULL)
  56.   return ;
  57.  
  58.  CcSetReadAheadGranularity = (FCcSetReadAheadGranularity)GetProcAddress(hProc, "CcSetReadAheadGranularity");
  59.  
  60.  if (CcSetReadAheadGranularity == NULL)
  61.   return ;
  62.  
  63.  s1.a2 = ;
  64.  s2.a7 = &s1;
  65.  
  66.         // After this call, ret must be 0x666, the given 2nd argument minus 1
  67.  ret = CcSetReadAheadGranularity(&s2, 0x667);
  68.  
  69.  if (ret != 0x666)
  70.   return EMULATOR_DETECTED;
  71.  else
  72.   return EMULATOR_NOT_DETECTED;
  73.  
  74. }

This technique(s) works in the 3 emulators I tested (Norman Sandbox, IDA+Bochs and Wine) and I’m pretty sure that them will work in any emulator.

Old Features

In the old –good?– days of MSDOS and Windows 9x the AUX, CON, and other special devices were used to read data from the keyboard, change terminal colors, etc… This behavior, while not currently supported (if I’m not wrong), works in current Microsoft Windows operating systems but not in emulators. The following is an easy example:

FILE *f;
  1.  
  2.     f = fopen("c:\\con", "r");
  3.  
  4.     if (f == NULL)
  5.         return EMULATOR_DETECTED;
  6.     else
  7.         return EMULATOR_NOT_DETECTED;

The unique “emulator” that simulates correctly this behavior is Wine. This technique was found by 2 of my co-workers, nick-namely, “PE_Luchin” and “Shaddy”.

Assembly

Emulating corrrectly a complete CPU is a very hard task and is also the most error prone area to look for incongruencies. Norman Sandbox works remarkably bad in this sense: The emulator fails (or it failed, I didn’t tested it since last year) with instructions like ICEBP or UD2 and allows changing, in example, the debug registers via privileged instructions. Easier to see in the following 4 examples:

int test1(void)
  1. {
  2.     try
  3.     {
  4.   __asm
  5.   {
  6.    mov eax, 1
  7.    mov dr0, eax
  8.   }
  9.     }
  10.     catch(…)
  11.     {
  12.         return EMULATOR_NOT_DETECTED;
  13.     }
  14.  
  15.     return EMULATOR_DETECTED;
  16. }
  17.  
  18. int test2(void)
  19. {
  20.     try
  21.     {
  22.   __asm
  23.   {
  24.    mov eax, 1
  25.    mov cr0, eax
  26.   }
  27.     }
  28.     catch(…)
  29.     {
  30.         return EMULATOR_NOT_DETECTED;
  31.     }
  32.  
  33.     return EMULATOR_DETECTED;
  34. }
  35.  
  36. int test3(void)
  37. {
  38.     try
  39.     {
  40.         __asm int 4
  41.     }
  42.     catch(…)
  43.     {
  44.         return EMULATOR_NOT_DETECTED;
  45.     }
  46.  
  47.     return EMULATOR_DETECTED;
  48. }
  49.  
  50. /** Norman Sandbox stoped execution at this point 🙁 */
  51. int test4(void)
  52. {
  53.     try
  54.     {
  55.         __asm ud2
  56.     }
  57.     catch(…)
  58.     {
  59.         return EMULATOR_NOT_DETECTED;
  60.     }
  61.  
  62.     return EMULATOR_DETECTED;
  63. }
  64.  
  65. /** Norman Sandbox stoped execution at this point 🙁 */
  66. int test5(void)
  67. {
  68.     try
  69.     {
  70.         // icebp
  71.  __asm  _emit 0xf1
  72.     }
  73.     catch(…)
  74.     {
  75.         return EMULATOR_NOT_DETECTED;
  76.     }
  77.  
  78.     return EMULATOR_DETECTED;
  79. }

These tests were launched against Wine, IDA+Bochs and Norman. While they don’t work in Bochs they makes failing both Norman Sandbox and Wine; both thinks the process has crashed and stops execution.

Conclussion

There are a lot of antiemulation techniques and these are just simple examples; writting much more elaborated ones is a matter of time and it’s simply impossible to circunvent all the antiemulation techniques. The old cat & mouse game 🙂

Mastodon.social Twitter Facebook Google+
# antiemulation # Malware