Greetings from Seattle! This is the start of a multi-part series about what I\u2019ve learned about Visual Studio debugger addins while writing the FNameAddin<\/a>.<\/p>\n First, however, we need to take a minute to discuss some basic stuff that most engineers I\u2019ve met are aware of, but few bother to dig into. The autoexp.dat<\/em> file is a peculiar beast in that you are encouraged to add to it, yet it is squirrelled away deep in the Visual Studio installation folder. It can be customized for your project, but has to be shared among all projects. All in all, it seems poorly thought out. We\u2019ll blast through the really basic stuff first, so anyone who is seeing this for the first time should read up on the supplementary links.<\/p>\n Deep in the Visual Studio install directory, at <VisualStudioInstall>\/Common7\/Packages\/Debugger<\/em>, there lives the autoexp.dat<\/em> file. It\u2019s just a text file, so go ahead and open it up and take a look around if you\u2019ve never done that before. This is where you can give the Visual Studio debugger some hints as to how you want your data to be displayed in the watch window. There are three sections: [AutoExpand]<\/span>, [Visualizer]<\/span>, and [hresult]<\/span>. We\u2019ll only deal with the first one for the time being. <\/span><\/p>\n The AutoExpand section is fairly simple and quite well documented at the top of the file, so I won\u2019t bother with it here. Sufficed to say, you can crack lots of simple data types by just adding them to the mix and possibly using one of the type suffixes. Go ahead and play around with this stuff. It\u2019s very safe in that it won\u2019t destabilize Visual Studio if you get it wrong, the debugger reloads the file every time you start a new debugging session, so feel free to edit and play around – just make a copy of the original file before you start. It\u2019s always nice to revert in a pinch.<\/p>\n I\u2019m far more interested in covering the $ADDIN-dlls functionality that is briefly mentioned. The file recommends looking at Microsoft\u2019s EEAddin sample in order to get started, but you should use some caution. It turns out that EEAddin has been broken for as long as I can remember and will crash if you use it as-is \u2013 how unfortunate! So let\u2019s take a different route. Let\u2019s make our own sample: MT_Addin.dll. The process looks like this.<\/p>\n You can download my sample project here<\/a> (3k)<\/p>\n Ok, now let\u2019s see what we have. Starting with the cpp file, we have our handler function with the following signature:<\/p>\n There are a few things to note here. First, we declare the entry point as WINAPI which is just a #define for the __stdcall<\/span> calling convention. This is what\u2019s missing from the EEAddin sample that causes it to crash. Next, notice that we don\u2019t use the dwAddress or bIsUnicode arguments. It might seem a little strange given that the whole point of this exercise is to look up addresses in the target process, but these are really just legacy arguments.<\/p>\n The most important thing here is the DEBUGHELPER<\/span>. Strangely, you have to define the type for yourself even though the format isn\u2019t under your control. (You can find it in MT_Addin.h<\/em>) Fortunately, it\u2019s not that complex and contains very little nuance.<\/p>\n <\/p>\n Our example doesn\u2019t do any memory queries at all, but it does create a simple string to tell you it\u2019s working.\u00a0 Finally, it returns S_OK for success \u2013 your addin should always return S_OK even when something goes wrong. When data can\u2019t be interpreted correctly, you should return success and set the output string to \u201cError processing the data!\u201d because returning an error will only display \u201c???\u201d in the watch window. Think of the return code to mean \u201cThe addin succeeded\/failed\u201d instead of \u201cprocessing this data type succeeded\/failed\u201d<\/p>\n In order for Visual Studio to find your DLL, we have two options \u2013 specify an absolute path in the autoexp.dat<\/em> file or copy our DLL into the \u201c<VisualStudioInstall>\\Common\\IDE<\/em>\u201d folder. I recommend the former method because it prevents pollution of the Visual Studio folder, and it facilitates debugging the addin.<\/p>\n In order to test the addin, we need a second project that contains a type called \u201cMyType\u201d. At this point, it really doesn\u2019t matter what MyType contains since our addin doesn\u2019t actually query process memory. We just need the match the type name in order to invoke our handler – so go ahead and try it. If you put a MyType variable in the watch window, it should display, \u201cHandled Data Type!!\u201d Don\u2019t worry, we\u2019ll eventually make some memory queries.<\/p>\n Debugging these sorts of addins is fairly straight forward. Edit the addin project\u2019s Debugging properties so that the Command item points to DevStudio IDE binary \u2013 it lives at \u201c<VisualStudioInstall>\\Common\\IDE \\devenv.exe<\/em>\u201d. When you run, your DLL won\u2019t be loaded, so your breakpoints are invalid initially. Don\u2019t worry about it though. The debugger will load your dll just in time and your breakpoints will go active. In fact, it will load and unload your any time it it\u2019s required.<\/p>\n Ok, that\u2019s it for now. I think we have the basics pretty well covered. There\u2019s a bit more to do with the standard addins, but hopefully things get a bit more interesting when we get to the Visual Studio extensibility framework.<\/p>\n","protected":false},"excerpt":{"rendered":" Download the source code for this post (3k) Greetings from Seattle! This is the start of a multi-part series about what I\u2019ve learned about Visual Studio debugger addins while writing the FNameAddin. First, however, we need to take a minute to discuss some basic stuff that most engineers I\u2019ve met are aware of, but few […]<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[5,3,8],"tags":[],"class_list":["post-12","post","type-post","status-publish","format-standard","hentry","category-debuggin","category-engineering","category-visual-studio","category-5-id","category-3-id","category-8-id","post-seq-1","post-parity-odd","meta-position-corners","fix"],"_links":{"self":[{"href":"https:\/\/www.myopictopics.com\/index.php?rest_route=\/wp\/v2\/posts\/12","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.myopictopics.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.myopictopics.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.myopictopics.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.myopictopics.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=12"}],"version-history":[{"count":80,"href":"https:\/\/www.myopictopics.com\/index.php?rest_route=\/wp\/v2\/posts\/12\/revisions"}],"predecessor-version":[{"id":171,"href":"https:\/\/www.myopictopics.com\/index.php?rest_route=\/wp\/v2\/posts\/12\/revisions\/171"}],"wp:attachment":[{"href":"https:\/\/www.myopictopics.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=12"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.myopictopics.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=12"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.myopictopics.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=12"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}AutoExp.dat<\/h2>\n
\n
\n
\n
\r\nADDIN_API <\/strong> HRESULT <\/em> WINAPI<\/strong><\/span> HandleMyType<\/span><\/strong>( DWORD <\/em><\/span>\/*dwAddress<\/span>*\/\r\n , DEBUGHELPER<\/em> <\/span>* pHelper<\/span>\r\n , int<\/em> <\/span>nBase<\/span>\r\n , BOOL<\/em> <\/span>\/*IsUnicode<\/span>*\/\r\n , char<\/em> <\/span>* pResult<\/span>\r\n , size_t<\/em> <\/span>maxResult<\/span>\r\n , DWORD<\/em> <\/span>\/*reserved<\/span>*\/ );\r\n<\/code><\/pre>\n\n\n
\n dwVersion<\/span><\/strong><\/td>\n \u2013 Structure version – <0x200000 for VS6, >=0x20000 for VS7 and beyond<\/td>\n<\/tr>\n \n GetDebugeeMemory<\/strong>()<\/span><\/td>\n \u2013 Useful for VS6 otherwise deprecated. Doesn\u2019t support 64-bit pointers<\/td>\n<\/tr>\n \n GetRealAddress<\/strong>()<\/span><\/td>\n \u2013 Gets the 64-bit value of the variable to be inspected. (Replaces dwAddress)<\/td>\n<\/tr>\n \n GetDebugeeMemoryEx<\/strong>()
\n<\/span><\/td>\n\u2013 Query raw memory from the target process.<\/td>\n<\/tr>\n \n GetProcessorType<\/strong>()<\/span><\/td>\n \u2013 Its use seems clear enough, but it always returns 0 for me.<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n Installing, Testing, and Debugging<\/h2>\n
Next Time<\/h2>\n