Geeks With Blogs
Remi Sabourin's Blog Architecture, Management and everything else

The Challenge

Imagine a development shop that maintains 100 .Net solutions, each composed of multiple projects. Imagine roughly half of these being shared domain services or shared common libraries, the other half being end web applications servicing multiple large clients.  Imagine any given project having the ability to reference classes/methods on any suitable re-usable service.   Imagine trying to get a grip on it all - a great architectural paradise or a great architectural nightmare.  Welcome to my world!

For a while I have been thinking about a way to manage dependencies.  Dependencies hold us back, they make us scared to change things.  But when we don't evolve, we don't improve, we spend time doing complete rewrites or adding layer over layer of code bloat and we don't refactor to achieve enough of my favorite thing..... (c'mon guys, you know it's coming).... RE-USE. 

I am dreaming of a searchable database of dependencies, most it being automatically generated and updated, although some could be specified manually if indirect.  Basically a master tool that would spider our build and production server(s) and document dependencies to any artifact - code, databases, folders, etc.   If we want to remove/update/rename anything, we can be confident in our actions.  It would point out orphans that we can clean up so as to keep our systems/code clean.  It would plug into an automated build process and send dependency reports and alerts.

The Research

Voice inside my head says "Start with the code".  I want to know for every method/property what method/property calls it in any project. Here's what has transpired so far:

First I figure a little app that calls the .Net framework could be in order, that spits the results to a database.  I know System.Reflection has metadata on dlls built in. On further inspection, System.Reflection only exposes things in the manifest, which does include assembly references (a start), but does not provide an insight into method references.  Next I look at the CodeDom namespace - if similar to XML DOM, I could create a solution to parse the source code and I could iterate the dom to find function calls.  In fact, there is a CreateParser() method - too bad for me there's no implementation. Maybe in a future .Net version?

I took a look at NRefactory that is an API that comes with the SharpDevelop IDE but can't find much documentation on this to see if it could do the job.

Examined some out-of-the-box add-ins/tools, NDepends being a possibility.  I figured I would have an automated build output an XML file from NDepends and then I could have code that parses it for the dependencies and puts them in a database..  NDepends looks like a good tool with some interesting features and a query language, but it does not output method dependencies (only type and assembly ones), although you can see them if you drill down using the application.  There is also licensing per individual.

I thought dlls could maybe be reversed engineered; there is a tool that comes with .Net called ildasm.exe.  Its output though is intermediate language code which I'm not familiar with and would be kindof brutal to parse.  I found out Reflector has a "hidden" API if you reference the EXE.   A nice MSDN article that has a promising name by James McCaffrey: http://msdn.microsoft.com/en-us/magazine/cc163641.aspx. May try this out, does not seem like the perfect high-level API but may not be too complicated knowing what exact statements to look for in intermediate language.

Then found out that FxCop also has a "hidden" API if you reference the EXE. Here's a nice doc Jason Kresowaty created to document this API: http://www.binarycoder.net/fxcop/pdf/fxcop.pdf.  This looks like the abstract API I was looking for.  This is my best bet right now.

The Execution

Concluding my research, my goal in the next few weeks is to:

- Use System.Reflection to log all the assembly references found in dlls

- Use FxCop or Reflector API to log all the method references found in dlls

- Log all the source code in the database (if this is impractical, use indexing service) so I can query it.  What the hey, why not use SQL Server's scptxfr.exe utility and log all the stored procs/views/triggers too.

- Have a GUI for the development teams that displays dependencies between assemblies and allows you to drill down to methods.  Have GUI that allows you to search for ad hoc strings in the source code..

I'll let you know how it turns out.

 

P.S. If you know of some wicked tool that can do this for me and allow me to RE-USE and conserve time, let me know!

Posted on Thursday, July 24, 2008 2:09 AM Architecture , Cool tools | Back to top


Comments on this post: Dependency Management

# re: Dependency Management
Requesting Gravatar...
http://msdn.microsoft.com/en-us/library/system.reflection.methodbody.aspx

System.Reflection namespace

Provides a glimpse inside methods to see what they are referencing, and that includes exception information. Basically you get a token tree.

This is not a total answer, but is one more piece in the System.Reflection jigsaw puzzle .
Left by Shaneo on Jul 24, 2008 12:53 PM

# re: Dependency Management
Requesting Gravatar...
I took a look, looks like it could help me get info on local variables. So that could help point out type dependencies. But at a more fine grained level, I need to see the method call statements to know dependencies on functions (this includes shared functions and functions in modules which do not involve any local variable being declared)
Left by Remi on Jul 24, 2008 3:54 PM

# re: Dependency Management
Requesting Gravatar...
NRefactory is a C# parser (similar to CodeDom) - basically it'll tell you the code calls a method named X, but you cannot easily know in which class that method is or which overload is being called - NRefactory just parses the syntax, you don't get any semantic information.
While you could get that using ICSharpCode.SharpDevelop.Dom, it's much easier (and more accurate) to analyze the compiled assemblies.
You should take a look at Mono.Cecil - it's a library to read (and modify) .NET assemblies, including the IL opcodes and associated method references.
Left by Daniel G. on Jul 25, 2008 11:09 AM

# re: Dependency Management
Requesting Gravatar...
Had time to check out your suggestion Daniel. Thank you very much! No documentation really on the API, but through the object browser, debugger and newsgroups, this looks like it will do the job and more! Thanks again for your insight, very much appreciated
Left by Remi on Aug 09, 2008 4:15 PM

# re: Dependency Management
Requesting Gravatar...
Quite inspiring,

looks great, but i have trouble understanding this, will come back tomorrow and try again lol. but good work.. its an important subject..

Anyway, thanks for the post
Left by web development company on Oct 08, 2009 6:48 AM

Your comment:
 (will show your gravatar)


Copyright © Remi Sabourin | Powered by: GeeksWithBlogs.net