Newbie Win32 Development Using Python
First a disclaimer. I’m not in anyway shape or form an experienced win32 systems programmer. I’ve always done application development or systems administration scripting. That in-between area where you get out your C/C++ compiler and start dealing with pointers is completely alien to me. This has created problems in the past when I need to do something outside of the scope of C# and found myself staring at MSDN docs in C trying to extrapolate the equivalent C# code, and typically being disgusted at having to use hand rolled Structs with calls to Marshall.GetLastWin32Error().
Worse still I end up with a nasty implementation that requires slow, fragile integration tests to verify behavior or very verbose mirror interfaces where I’m testing order of calls. Look at the following code sample to read a reparse point’s target directory from code project:
Int32 outBufferSize = Marshal.SizeOf(typeof(REPARSE_GUID_DATA_BUFFER));
IntPtr outBuffer = Marshal.AllocHGlobal(outBufferSize);
try
{
// Read the reparse point data:
Int32 bytesReturned;
// WOW what a signature, it’s requiring blank data!
// Looking at MSDN the signaure contains input and output buffer and size
// for both, meaning I’m always sending something as IntPtr.Zero
Int32 readOK = DeviceIoControl( hFile,FSCTL_GET_REPARSE_POINT, IntPtr.Zero, 0,
outBuffer, outBufferSize, out bytesReturned,IntPtr.Zero);
if (readOK != 0)
{
// Get the target directory from the reparse
// point data:
REPARSE_GUID_DATA_BUFFER rgdBuffer =
(REPARSE_GUID_DATA_BUFFER)Marshal.PtrToStructure(outBuffer, typeof(REPARSE_GUID_DATA_BUFFER));
targetDir = Encoding.Unicode.GetString( rgdBuffer.PathBuffer,rgdBuffer.SubstituteNameOffset,
rgdBuffer.SubstituteNameLength);
</p>
//removed a bunch more stuff to format the result
// Free the buffer for the reparse point data:
Marshal.FreeHGlobal(outBuffer);
</div> </div>
In the past when building these kind of things for less complicated code I just would mirror out all the untestable code like Marshal , DeviceIoControl and CloseHandle calls with interfaces, then create a production wrapper. This however leads to very long tests, with lots of specifics about the internals of the interaction, in the above example the support code looks like:
{</p>
void FreeHGlobal(IntPtr buffer);
Object PtrToStructure(IntPtr pointer, Type structure);
Int32 SizeOf(Type structure);
IntPtr AllocHGlobal(int cb);
}
public class ProdMarshal : IMarshal
{
public void FreeHGlobal(IntPtr buffer){
Marshal.FreeHGlobal(buffer);
}
// you get the idea from here
} </div> </div>
Not terribly pretty mocking and testing this leads to is it? This isn’t also counting the win32 calls that are needed to setup and finalize the data around this. Before its all said and done to make this “testable” you end up with 2 interfaces, 2 wrapper production classes, and very heavy interaction specific mocking to even handle this basic idea of getting a value from a file system object. Before you try and point out “well you can reuse that logic later” look at the size of Marshal and ask yourself do you really want to mock all that out?
Adding insult to injury your mocks have to be very good to replicate the behavior of the actual real thing, and it varies greatly across platforms. Windows 7 and server 2008 r2 allow reparse points (or symlinks in the this case) from directory to directory, but Vista and 2008 r1 are a no go and will fail.
No wonder the systems developers that I know completely turn there nose up at unit testing and instead focus on slow integration or systems tests. The abstractions take time to make happen and the code itself is just not testable without lots of mirrored interfaces.
So not wanting to spend all my time just writing mirrored interfaces day in and out for win32 calls, nor liking the idea of turning my back on unit testing, I decided to take a look at dynamic languages win32 support.
First I looked at Ruby and the win32 libraries I saw didn’t give me the above functionality I needed with the minimal amount of hassle I wanted. DeviceIoControl up there for example with its stunningly bad signature is still pretty rough in Ruby as they have a one to one signature mapping.
Python however has had for a very long time strong win32api support and removes some of the awful pain that you have to deal with. Below is the equivalent of the code above, again with items stripped out for brevity before and after it, but they’re roughly computationally equivalent and the Python version wins out more the more I include.
#notice how I don’t have to pass in size of the buffers now. I still have the need to
# pass None in for the input buffer here however
buf=win32file.DeviceIoControl(h, winioctlcon.FSCTL_GET_REPARSE_POINT,None,OutBuffer=output_buf, Overlapped=None)
fixed_fmt=‘LHHHHHH’
fixed_len=struct.calcsize(fixed_fmt)
tag, datalen, reserved, target_offset, target_len, printname_offset, printname_len =
struct.unpack(fixed_fmt, buf[:fixed_len])
It’s still not pretty, but its briefer, and I can actually stub out or test struct.calcsize, AllocateReadBuffer and DeviceIoControl with no need to mess around with setting up lots of test harness code. Using my current mocking framework the test code looks something like this:
buffer = object()
alloc.stub(buffer)</p>
#removed extra setup code for calcsize and DeviceIoControl
getTargetDir()
mock.assertCalled(win32file, “AllocateReadBuffer”) </div> </div>
Much cleaner, much lower friction and so I’m satisfied with Python for win32 development so far. I’ll give C# another look when I have the dynamic keyword and I don’t have to make explicit mirror interfaces for everything that’s untestable.