The Etch Call Graph Profiler (version 1.0b)

Frequently Asked Questions


This document provides answers to some common questions that come up with the Etch Call Graph Profiler.

Contents


System and program considerations for the Etch Call Graph Profiler


Question: What are the system requirements for running the Etch Call Graph Profiler?

Answer: To run Etch Call Graph Profiler, you'll need Windows NT 4.0, at least 32 MB of physical memory, and a large swap file. Your computer also needs to have a CPU that supports the RDTSC instruction for accessing the hardware cycle counter. For Intel processors this corresponds to Pentium or later processors.


Question: Does Etch work under Win95? The instructions say that Etch works under NT. I run Win95. Can I use Etch there?

Answer: Yes, it should work, but it's even less supported than Etch on NT.


Question: What kinds of binaries can I instrument?

Answer: You can instrument 32-bit PE format Windows binaries, including executables and DLLs. Certain kinds of instruction sequences found in hand-written assembly language will fail when instrumented, but these sequences occur rarely in practice. Examples include self-modifying code, code that jumps into the middle of an instruction, and code that uses pointers to instructions to refer to data.

There are a small number of system DLLs that cannot be instrumented, such as user32.dll and kernel32.dll. The reason for this is that the operating system has hard-wired entry points to procedures in these DLLs. The Etch Call Graph Profiler handles these DLLs appropriately, and hence you do not need to do anything special for applications that use them.


Question:What applications have you tried the Etch Call Graph Profiler on?

Answer:The Etch Call Graph Profiler has been used to instrument many commercial applications: Microsoft SQL server, Microsoft Monster Truck Madness, Microsoft WinWord, Microsoft PowerPoint, Microsoft Money, Microsoft Schedule, Microsoft Internet Explorer 3.02, Adobe Photoshop 4.0, Adobe PageMaker, Lotus Word Pro, Netscape Navigator 3.01, etc...


Question:Does the Etch Call Graph Profiler work on 16 bit code?

Answer: No. Etch will not instrument a module unless it is 32-bit code in PE format.


Question:Does the Etch Call Graph Profiler give meaningful information for recursive procedures?

Answer: Yes. Recursive procedures are indicated in profile output with a '*' next to the procedure name. For more information on how the Etch Call Graph Profiler handles recursive procedures, see "Profiling Recursive Procedures" in the Etch Call Graph Profiler documentation.


Question: Can the Etch Call Graph Profiler be used with multithreaded applications?

Answer: Yes. The Etch Call Graph Profiler correctly maintains time for multithreaded applications. Profile time is only credited to one thread at a time. More precisely, if one application thread is suspended while another thread in the same application is running, only the running thread will accumulate profile time. Put another way, for a uniprocessor, the sum of the profile time for all the threads should be equal to the total execution time of the application.

The Etch Call Graph Profiler combines activity for multiple threads when reporting results. If you would be interested in per-thread profile reports, or selective combining of thread profiles, please send feeback to etch-support@etch.cs.washington.edu


Question: Can I use CGProfile on a multiprocessor?

Answer:CGProfile uses the Pentium hardware performance counter. On multiprocessors, there are multiple hardware performance counters. Unless they are synchronized, CGProfile will not give accurate results.

Although CGProfile will provide output for multiprocessors, be aware that the timing info given may be substantially distorted. If you are interested in a version of CGProfile with multiprocessor support, send mail to
etch-support@etch.cs.washington.edu


Question: What is the modules.bdb file?

Answer: The file modules.bdb contains information that the Etch Call Graph Profiler maintains about modules used by an instrumented application. You should NOT add modules to this file by hand, as this will usually cause the instrumented application to fail to run. To find out more about the module database, see "Managing Etch Call Graph Profiler Projects" in the Etch Call Graph Profiler documentation.


Question: What additional link flags do I need to specify when using MSVC 5.0 so that I can produce object files that the Etch Call Graph Profiler can process?

Answer: Etch needs relocation information to rewrite a binary. Most compilers produce this by default, including pre-5.0 releases of Microsoft Visual C. MSVC 5.0, though, by default, does not produce relocation information for .exe files. If you are invoking the linker from the command line or maintaining your own makefile, you will need to add the flag "/fixed:no" when you invoke link.exe. If you are using a Microsoft Developer Studio project to build your application, then you need to add the flag to your project settings:

For compilers other than MSVC 5.0 please consult the documentation for the compiler.


Question: I am using MSVC 5.0. How do I create an executable that contains both relocation records and debug information. Furthermore, how do I create a call graph profile of it?

Answer: Here are the steps that you need to take in order to produce an executable and to create a call graph profile. Example below assumes that the source file is hello.c.


Question: How do I instrument a program that relies on multiple executables (.exe files)?

Answer: The key in getting this to work is that these applications typically need to share copies of one or more DLLs. This means that instrumenting the executables in different project directories would not work, as each project directory will contain its own private copy of the instrumented DLL. You can cause the executables to share DLLs simply having them share the same project directory.

Note: In the current version of the Etch Call Graph Profiler, you will only get output for the last executable to terminate.

Suppose we have two programs, helper.exe and master.exe, where running master.exe causes the execution of helper.exe as a helper process. Frequently, the two programs communicate through common DLLs. Etch must determine the set of DLLs used by both helper.exe and master.exe. The specific steps we need to follow for helper.exe and master.exe are as follows (obviously, you replace these names with the names of your programs):

  1. Create and cd into a project directory.
        mkdir myproject
        cd myproject
    
  2. Ensure that helper.exe is not already running. On Windows NT, you can use the Task Manager to find and terminate helper.exe if it is running.
  3. Run cginstrument.exe on master.exe.
        cginstrument master.exe
    
  4. Run cginstrument.exe on helper.exe.
        cginstrument helper.exe
    
  5. Etch has now found and instrumented all the DLLs that are statically loaded by master.exe and helper.exe. You are now ready to run your experiment. In general, master.exe will not be able to start helper-etch.exe automatically, so you need to start it yourself before starting master.exe.
        start helper-etch.exe
        start master-etch.exe
    
    As the Etch Call Graph Profiler learns of additional DLLs used by the two executables, they are instrumented on-the-fly.

  6. When you are done running master.exe, profile output can be found in cgmon.output. Rename this file to cgmon-master.output. Now terminate helper-etch.exe using the task manager. This will generate a new cgmon.output for helper.exe. Rename this file to cgmon-helper.output.
  7. Postprocessing:
        cgprofile.exe -mon cgmon-master.output > master.profile
        cgprofile.exe -mon cgmon-helper.output > helper.profile
    
Example: Here are the steps you need to use to instrument Monster Truck Madness (Trademark of Microsoft), which uses ddhelp.exe. The example assumes that Monster Truck Madness has been installed into the directory c:\monster, and that ddhelp.exe lives in c:\winnt40\system32 directory.


Question: I want to profile a DirectDraw application. What do I need to do?

Answer: DirectDraw applications use two executables. Use the steps given above for Monster Truck Madness, substituting the name of your application for monster.exe.


DLLs

Question:How are instrumented files named?

Answer: The Etch Call Graph Profiler transforms every DLL used by the program, say "foo.dll", into either "foo-patch.dll" or "foo-etch.dll". The "-patch" DLL is created when the Etch determines or the user specifies that foo.dll is not to be instrumented. Although the code for the DLL is not changed, Etch modifies the imports used by the DLL. The "-etch" DLL is created when Etch Call Graph Profiler modifies the code within the specific DLL.


Question: My application loads many DLLs dynamically with the Win32 LoadLibrary call. How are these DLLs handled?

Answer: When the Etch Call Graph Profiler learns of a new DLL that your program uses, it will automatically instrument it for you. This is called on-the-fly instrumentation, and its discussed in more detail in Call Graph Profiling in Four Easy Steps.


Problems when instrumenting with cginstrument.exe

Question: When I try to run CGInstrument on my application, it fails with the following message:
    FAILURE: Could not patch foo.dll Exiting
What does this mean?

Answer:This message means that Etch was unable to read the DLL import tables for foo.dll. The best thing to do in this case is to take a look at the instrument.warning log, which should have a more information on why Etch had trouble. Below are some possible error messages, and their causes.


Question:When I try cginstrument.exe on my application, it fails with the following message:

    <exename> is not a PE executable.  Can't continue.
What does this mean?

Answer:The Etch Call Graph Profiler only works on 32-bit 80x86 Windows NT binaries in PE format. This message means that one of the modules your applications appears to use is either not a valid executable, or is in some other executable format (such as a 16-bit executable.)


Problems when running a profiled application

Question:I am following the instructions for instrumenting notepad.exe, but when I run the instrumented version (notepad-etch.exe), I get the following error message:
"The application failed to initialize properly (0x0c00000005). Click on OK to terminate the application."
What is wrong?

Answer:If you have downloaded the Internet Explorer 4.0, comctl32.dll has been upgraded to a new version. Etch does not currently handle the new DLL correctly. We are working on a fix for this, and meanwhile you can do the following to get around this problem:

  - open up the modules.bdb file in notepad (in the instrumentation directory)
  - find the entry for COMCTL32.DLL.  It should look something like:
 
module COMCTL32.DLL
 	path C:\WINNT\SYSTEM32\COMCTL32.DLL
 	index 3
 	...
 	codediscovery 2
 
  - change the line: 
 	codediscovery 2 
    to 
 	codediscovery 0
  - Instrument the code again, using the flush flag:
 	cginstrument /flush notepad.exe
  - And then try running the instrumented code again:
 	notepad-etch.exe


Question:I have instrumented my application program, but when I run the instrumented version I get a run-time failure. What do I do next?

Answer: There are a number of potential problems that can cause an instrumented program to fail. Here, we describe some of the problems and possible remedys.

Some specific problems


Question:I instrumented a program and it runs fine, but I see some failures in the instrument.warning log file. Why is this?

Answer:If you see some failures messages in the instrument.warning, but don't see any failure messages in cginstrument.log, there is usually nothing to worry about. Failure messages in instrument.warning can occur when Etch has problems instrumenting a specific module. When such problems occur, Etch automatically tries changing the instrumentation parameters to correct the problem. When failure messages appear in the instrument.warning log but not in the cginstrument.log, it means that Etch found a set of instrumentation parameters that allowed it to process the module.


Question:My instrumented application runs, but it gets an error at runtime. What should I do?

Answer:First use the debugger to determine which module of the instrumented program is causing the runtime error. One way to do this is to look at the call stack of the instrumented program and see which module caused the error. Once you have identified this, modify the properties of the module that is causing the error. For information on how to do this, see the section Editing the module database in "Managing the Etch Call Graph Profiler Projects".

Once you have identified the problem module, try reducing the codediscovery parameter (from 2 to 1 or 0) to see if this eliminates the specific error you saw. If the instrumented program still fails in the same module, change the etchmoduletype property for the module to patch. Repeat these steps until you eliminate all runtime errors.

We would like to hear from you about applications/modules that have these sorts of problems. If you discover a module that can't be instrumented with etchmoduletype etch and codediscovery 2 please send a note to etch-support@etch.cs.washington.edu


Question:I instrumented an application, then modified the application and re-ran cginstrument. However, the Etch Call Graph Profiler seems to be usingthe old version of the program. What went wrong?

Answer:To save on instrumentation time, the Etch Call Graph Profiler saves instrumented versions of application EXEs and DLLs. When you re-instrument an application, cginstrument.exe will use the cached version. To insure that the Etch Call Graph Profiler creates a new, up-to-date instrumented module, you should use the '/flush' flag to cginstrument This will cause all the modules to be re-instrumented. If you have lots of modules, and don't want to re-instrument all of them, you can delete the instrumented version of the module you modified and run cginstrument without the /flush flag. It will noticed there is no saved copy of the instrumented module and generate a new one based on the new version of the module.


Question:I instrumented an application, then modified the application and re-ran cginstrument. Now the Etch Call Graph Profiler won't instrument it anymore. What happened?

Answer:The module database (modules.bdb) might be inconsistent with the new version of your application. Use the '/flush' flag to cginstrument to make sure that all the modules are re-instrumented. If even after you do this, you encounter problems, delete the module database (del modules.bdb), and try again.


Question:I tried to follow the hints above but my problem doesn't seem to be covered by the FAQ. What should I do?

Answer:Please send us mail if you can't get our profiler to work with your application. We are interested in your problems, and although can't promise to provide support, we will try. Direct support requests to etch-support@etch.cs.washington.edu. Please include the name and version information for your application, hardware and software configuration information for you system (including Windows NT version and service pack level), a description of the problem, and a copy the modules.bdb file from your project directory.


Debugging

Question: Can I debug an instrumented program?

Answer: Etch does not maintain debug information when it instruments an executable. Consequently, symbolic debugging is not possible. Assembly-language debugging can still be useful.


Question: How can I set the debugger that is invoked when an application generates an exception and asks you to "click Cancel to debug."

Answer: You set the default debugger as follows:

  1. Run the registry editor
        regedt32.exe
    
  2. Find the key HKEY_LOCAL_MACHINE\Software\Microsoft\Windows NT\Current Version\Ae Debug
  3. change the value "Debugger" to the debugger you would like to use. For example, if you want to use msdev.exe from c:\msdev\bin, set "Debugger" to
        C:\msdev\bin\msdev.exe -p %ld -e %ld
    


Question: I suspect that my program is failing in DLL initialization code. How can I debug this kind of problem?

Answer: Using a debugger such as msdev.exe, you can step through the DLL initialization code. The hard parts are persuading msdev.exe to set a breakpoint early enough and figuring out where the DLL initialization routines actually are. Msdev.exe discards breakpoints set at arbitrary instructions between runs, so you have to find a place to set a breakpoint that has a symbolic name.

Once you've executed the program and hit this breakpoint, you can walk up the stack to find the instruction in NTDLL.DLL:LdrpRunInitializeRoutines@4. This routine invokes the DLL initialization code for each DLL that is loaded. Find the call instruction that invokes the initialization routine, and set a breakpoint. Now you can step into the entry point of each DLL.

Note that msdev.exe will discard breakpoints at call instructions between runs. If you absolutely need to see every DLL as it is loaded (i.e., including DLLs that were loaded before the call), you could set a breakpoint at the beginning of LdrpRunInitializeRoutines@4. This breakpoint should be preserved between runs; you may have to exit and restart msdev.exe.

Tip: Once you've discovered the PCs of LdrpRunInitializeRoutines@4 and of the special call instruction, remember them. They will remain constant as long as you're using the same version of NTDLL.DLL.


Copyright (c) 1997 The University of Washington. All rights reserved.