From Glue Logic to Program Logic
My first introduction to the Python computer language was in the late 90’s. The project I was on was being moved to “an evil empire” in Oregon and I was tasked with transferring the entire contents of our CR database into “something” that they could easily parse and/or import. At this time, Python was (as far as I was concerned) a great “glue logic” language. You wouldn’t use it to deliver your application, but you could use it for quick one-off tasks like building your code base, doing large scale search-and-destroy (or modify) file operations, etc. I used it to execute queries against the database and push the data into several output files.
Since then, I’ve worked on a few Python applications:
- SNMP MIB Browser, delivered as a downloadable program for Windows
- A tool to track requirement and test case changes in MS Word and handle traceability and user notification
- A unit-test suite for the code base of a dialysis machine
- A tool-chain for “building” the pattern/image sets that go onto an SD memory card in a pattern cutter.
- A lint processing system; zillions of code files in, one simple ranked report out.
There were numerous other small projects scattered throughout. All of these were 100% Python. I’ve even seen a company where their entire server based sensor system (thousands of input readings in real-time) is built almost entirely on a python framework. Python has evolved from being the “glue-logic” that helps the build process or the release process to the “program logic” that makes the application work.
IronPython
IronPython is an implementation of the Python language which runs on the .Net framework. That is to say, it is a collection of .Net assemblies that allow you to compile and execute Python code as a .Net language such as C#, Visual Basic, etc. As of the current release (2.6), you cannot use many of the existing Python (called cPython) libraries with it, as they are actually binary libraries (or rely on them), and cannot interface with IronPython. On the other hand, you can access any compatible .Net assembly. This gives you a (very) large space from which to draw capabilities that you use IronPython to glue together and solve the task at hand.
There are essentially three ways you can use IronPython. The first is that you can use it to create whole applications by itself. The second is to have a primary implementation in a .Net language (e.g. C#) as your main program logic and then use the IronPython engine (and Python code) to handle details that are dynamic. The third is for quick “knock off” scripts used to do system operations or small easy tasks.
I strongly recommend staying away from the first option. Before the flame wars begin and I’m branded as a heretic, you should recognize that while IronPython executes as a .Net language, the support for it in Visual Studio is minimal, at best. An add-on for using it in VS, IronPython Studio, hasn’t been updated in 2 years and the remarks on it are mixed at best. If you are working in .Net and you’ve worked in just about anything else for comparison, you know that intellisense, refactoring, and GUI development support in VS is pretty good. Without it, you’re guessing the names of imports, classes, functions, manually adding callbacks, fiddling with code (and recompiling) to get the GUI just right; that’s a hassle and more importantly, a waste of valuable time.
The second option, the one which this article focuses on, seems like a very good use of IronPython. Consider the following scenarios:
- You are delivering a business application to your end users. They each have a certain way they like the application configured. Or they each have a custom processing need (e.g. business logic). Your application is the engine that contains and performs the main operations on the data. You can use IronPython to create custom scripts for each customer to handle their particular configuration and business logic. When they need updates or modifications, you send them updated Python scripts.
- You have a server based application and users have a “client” on their desktop. The client application allows them to pull/push data to/from the server. You provide extra processing capabilities in the form of scripts that the users can either create or obtain from your organization.
- You want to write a video game. Using a scripting engine in video games is very common. All sorts of “operations” and “capabilities” (initialization, dialog, plot, victory condition, artificial intelligence, level instantiation, etc.) that are constantly changing during development not become scripted and can be handed off to non-programmers. The syntax of Python is relatively easy and you don’t have to recompile the engine every time you change the code.
The third option is on an “as-needed” basis. For these types of operations, I usually stick to cPython; the libraries are there and existing Python tools recognize the existing packages without any hassles.
Examples
The first thing we should do is ask ourselves how we think IronPython will be used. I created a list of operations/situations that I wanted to create an example for so that I could understand how the limits (and syntax) of using IronPython:
- Define a class in C#, access it from Python.
- Define a class in Python, create an instance and call a member function from C#.
- Execute a Python function (not the same as a class).
- Define a Python variables as module level variables, access and manipulate them from C#.
- Pass a function pointer (delegate) from C# to Python. Allow Python to call it with an argument.
- Execute a “yield” operation in Python.
- Trap a Python script compile error.
- Trap a Python script execution error.
I’m going to discuss some of these but most should be fairly obvious (and the comments in the code (download from here) should be sufficient to tie it all together.
Environment
- The entire application code was written in Visual Studio 2008 Professional edition. The code should run without an issue in the Express edition as well.
- I used IronPython version 2.6, which can be found here.
Test Harness
In order to execute the tests, I created a new project as a simple C# Windows Forms application. A screenshot of the finished product is below.
Executing IronPython Tests
The program is little more than a form with buttons for executing tests, a “SimpleLogger” class for logging messages, a timer updated for updating the logger, and the code for executing the tests themselves. The “Setup Python” button initializes the IronPython engine and scope (you can segment variables and data by “scope”). I left it in so you could see that it takes a non-trivial amount of time to initialize the engine.
Because the entire code base can be downloaded, I’m going to be sparse on including code here unless it will be value added to do so. I will walk through the first example, executing a C# class from Python, so that you get the feel for the flow of execution.
Infrastructure
The following statements include the IronPython and Microsoft Dynamic Language Runtime (DLR) namespaces.
using IronPython.Hosting; using IronPython.Runtime; using Microsoft.Scripting; using Microsoft.Scripting.Hosting;
I’ve also include the following declarations at the top of the main form class:
private ScriptEngine pyEngine = null; private ScriptRuntime pyRuntime = null; private ScriptScope pyScope = null; private SimpleLogger _logger = new SimpleLogger();
The SimpleLogger class is included in the source code. It is a thread-safe tool for creating log entries and works both from the C# and the Python side.
Initialization
The following bit of code initializes the engine and sets up the “scope”. The ScriptScope is the container that holds all the variables, functions, and class definitions that you would like to “group” together. When the engine executes, it executes against a scope.
if (pyEngine == null)
{
pyEngine = Python.CreateEngine();
pyScope = pyEngine.CreateScope();
pyScope.SetVariable("log", _logger);
_logger.AddInfo("Python Initialized");
}
Note that the _logger object is passed into the Python Scope by pyScope.SetVariable(…). Now Python has a reference to the SimpleLogger instance and can reference the methods it exposes.
I also created a simple function to take a string (Python code statements), compile them and execute them in the scope defined.
private void CompileSourceAndExecute(String code)
{
ScriptSource source = pyEngine.CreateScriptSourceFromString(code, SourceCodeKind.Statements);
CompiledCode compiled = source.Compile();
// Executes in the scope of Python
compiled.Execute(pyScope);
}
Now all you have to do is construct the Python code using strings, hand it off to the function, and it executes automatically.
For the first test, we want to add an element to the _logger from Python. First, we need to create the Python code. We can do this by loading it from a file or by just passing in the string. For this project, I’m going for a “self-contained” solution, so the Python code will be created by C# directly:
private String CreatePythonScript1()
{
String result = "";
string[] lines =
{
@"def DoIt1(logObj):",
@" logObj.AddInfo('Executed in a function call using log object input.')",
@"",
};
result = String.Join("\r", lines);
return result;
}
I’m being a bit anal and using the “@” for raw strings here. In Python, a string can be either enclosed in single quotes or double quotes (and even triple single/double quotes), but I’m going to stick with single quotes so the editor doesn’t start coloring/un-coloring and confusing me. This is a simple declaration of a Python function, “DoIt1(logObj)”, which takes an argument logObj.
When you hit the “Test 1″ button, the following code gets executed:
private void btnTest1_Click(object sender, EventArgs e)
{
CompileSourceAndExecute(CreatePythonScript1());
CompileSourceAndExecute("DoIt1(log)");
}
The first line compiles the Python function. The second passes in the _logger object, defined in the Python scope as “log”. And the list on the GUI gets “magically” updated.
The rest of the code contains the remaining examples. Unless you are looking for something fairly exotic, I think I’ve covered the majority of the examples for usage that I think are practical for working applications. Feel free to leave feedback if you have any trouble or need clarification.
The Source Code can be downloaded from this link: IronPythonTest2 Source Code