Cerrisian Blog

Freemind management of mnemosyne cards is an awesome idea! Create and organize subject matter in FreeMind, then export (leaves probably) to Mnemosyne cards with containing branches as categories. The tree could also be exported to a LaTeX memoir document, so it’s three in one: Tree, narrative, and notes. OOooh, I really like this idea.

• Freemind needs a: spellchecker, automagically updated file view and clone nodes.

• Use python 2.6 ast module to make pretty asserts that use the assert keyword.
• the ast module could also be used to implement something akin to the Eiffel “old” keyword. But I think a registry of expressions to evaluate before the method and are accessable by name e.g.
  • old(expr1=lambda s: s.attr, expr2=lambda s: s.method()) and accessable by old.expr1, old.expr2
• would be cleaner to implement.
• Cmockery is hugs and cuddles, now all I have to do is add it to our make file for OS, mwahahahah!
• Pythoscope equivalent for C would be interesting, and probably useful. Although really hard to implement…Whoever did it would become really good at C, though.
• Contracts for python need to handle functions as well as classes (perhaps through wrapper class with __call__ or some such?)
• Also looking at automake and autotools, makefiles don’t look as brain melting as they used to.
• Automatically generating Sequence diagrams based on unit test or code execution would be AWESOME, pyumlgraph already does it for class relationship diagrams (generally, at least)

What’d be really cool would be a firefox extension that allows you to highlight phrases, pops up a quick dialog box and then creates flashcards for mnemosyne. I think that this would basically give you all you needed for incremental reading. (With the addition of a bookmarking system, hey, Firefox already has that! :))

One possibility is to vaguely mimic the interface C4J uses for Java with class decorators and additional classes (possibly inside the defined class).

I’m not sure if it’s a good idea to separate the contracts from the implementing code…But it would add a lot of flexibility to be able to add contracts to existing classes (via wrapping).

Alright, so I just took a bunch of notes on my distributed system testing project in FreeMind. It’s really slick software for keeping trees of notes. For some reason it’s a lot easier to use than Leo for this type of stuff, partially because information is only stored in “Node Names” and partially because the tree is not one dimensional, this allows me to see a lot more at once. I’m really impressed at how nice the Java GUI interface is, it actually uses themes correctly!

The only big feature it’s missing is spellcheck. Maybe I can add it by using the Jazzy or jOrtho libraries.

I just realized a *much* better way to implement design by contract in python. Dynamic proxy objects. This should solve the infinite recursion/stack poking problem I currently have in my design by contract library. It also may make keeping track of previous state easier. I wonder if this is somewhat functionally equivalent to how Eiffel actually implements it…

Abstract

Testing distributed systems (such as a network) with automation is difficult. In this post I will attempt to codify a general approach to testing a distributed, heterogeneous system with automation. Some goals of this approach are:

• To be implementable on top of a “script based” automation system incrementally.
• To allow for easy and effective test failure isolation and troubleshooting.
• To enable simulations of the distributed system that help testers/engineers gain insight about how the distributed system works.
• To allow for automated or computer assisted test case generation.

Motivating example

You have a heterogeneous system of providers and clients. Providers give services to clients. Let’s say you have a few storage providers in your distributed system. You also have many pieces that are storage clients. When a storage provider gives storage to a client, it should have add the client to a list of clients that it has provided storage to. In addition the client should add the provided storage to a list of all the storage that has been provided to it. You also have to guarantee that the storage provided to the client is usable.

The Solution

To test this behavior you have a set of proxy objects that are manipulated by the test framework. The proxy objects then send the correct the commands to the actual pieces of the distributed system to provide the storage to the clients. These proxy objects have internal state that keep track of what state the distributed system should be in. Periodically (depending on performance constraints) the state of the framework proxy objects are compared to the state of the actual system. If they do not match, a test error is raised specifying what part of the state does not match.

• The expectations of a component should be grouped with the interface to the component.

Each proxy object has preconditions and postconditions associated with each of its methods. An invariant shared by all proxy objects is that their the internal state matches the state of the corresponding piece of the distributed system.

The proxy objects automatically poll the actual component and compare the actual state to the expected state to determine if the actual component is behaving correctly.

Some cool things we can do once we have proxy objects

• Specify certain states and state transitions that we require. We then use a constraint solver (either deterministic based on object contracts or heuristic based on simulator runs) to determine the optimal test execution based on a few things (Maximum allowable time, maximum number of components of each type, etc.)
• Get detailed reports of the exact components that failed in a test case
• Keep track of what components fail the most. If we keep track of the reasons that pieces failed, we can also determine what the most likely causes are automatically.

Simulation

To effectively simulate, the proxy objects interfaces to real devices must be disabled. This is best implemented as a a separation between the part of the proxy object that communicates with the real components and the part that keeps track of the expected internal state. Simulation would be implemented by only using the part that keeps track of expected internal state. The part that communicates with the actual object could either be disabled or replaced with a dumb object that does nothing and always return that the state matches.

Implementation Notes

• The proxy object doesn’t need to keep track of all the state of the real component. Detailed tracking can be added incrementally.

Notes and Gotchas

• What about issuing commands to components that fail preconditions? The component should go into a proper failure state. So preconditions shouldn’t prevent calling, but alert the user that preconditions aren’t met and update their expectations of the state of the component based on the preconditions being false. This is hard to do well with a clear API. Perhaps users should check preconditions manually? Or the object treats preconditions different based on a mode it is in?

What are the downsides?

Experts can take into account other things when determining optimal execution — this forces these pieces of knowledge to be codified into the system. It may be more efficient for these pieces of knowledge to be just used and not codified.

I can’t think of a clear and obvious way to detect hardware failures directly through the proxy objects. They would help isolate any such problems though.

Since I had some free time and wanted to unwind, I did an upside drawing of Riddick from the Pitch Black series of movies. I like how the hands turned out, but the relative proportion of the head and face is wrong. The glasses are alright, but need some work as well.

I have difficulty drawing long curves that don’t have neighbors that I can go off of. Some possible solutions might be to break up the large curve into multiple smaller ones or “make up” neighboring curves to go off of. The other thing I generally have trouble with is measuring spaces between curves, but I’ve been practicing and focusing on remember those. That may also help with dealing with curves that are alone.

I’ll have to do some more negative space drawings and get better at judging/remembering spaces between curves. I did alright on most of this upside down drawing, but it seems like this is my next major stumbling block.

Just means more negative space drawings for me

One thing I forgot to mention for troubleshooting is you need context and a mental model first. If you don’t have a good idea of what the system is supposed to do, then you can’t trouble shoot it. So, take some time and create a flow chart and diagram of the expected operation, this will give you the correct context to understand how the system works.

So, I had a discussion with my dad about automated troubleshooting and came up with some cool ideas: An automated guide to teach how to effectively troubleshoot. It would basically be a wizard typing thing that would go through some basic strategies, e.g.

• Make a  lit of straightforward/obvious things to check for errors: Is it plugged in, can all the equipment be pinged, is their fuel/ink in it, etc.
• Based on things checked, what could be wrong? Make hypothesis
• Determine how to test each hypothesis
• Test hypothesis, accept/reject hypothesis based on test. If necessary, refine hypothesis and test again (repeat)
• Problem is now isolated.

This could be expanded into a wizard to generate expert systems for automated troubleshooting, especially if it’s a wizard through a programming interface (something like a bash shell or command line interface). It could be really useful if we keep track of how often each problem turns out to be the problem, then we can use Bayesian inference to determine the most likely problems and which experiments/tests will narrow down our problem.

Of course, the most useful place to use this type of system would be a heterogeneous networked system (like the type of stuff I do at work). Yet another Master’s thesis idea…

Well, I’ve got a few more ideas for my Master’s thesis: Simulations/Discrete simulation. This may dovetail nicely with bayesian inference, genetic programming and scientific computing. I may also be able to figure out how to associate it with autodidactic software. It’d also probably be a good career choice to develop expertise in simulations.

I looked through the course catalog at UCCS for CS graduate classes and the things that stood out to me were computer learning algorithms and software engineering/design courses (Design By Contract, object oriented problem solving, etc.)

I’ll figure out a great thesis topic eventually. I definitely want to do something new and interesting, not just a review of existing literature.

Oh! Another potential for exploration is procedural content generation.

Well, that was a long day. I spent most of it at the lab working on senior project and operating systems with my group. My evening was spent going through my mnemosyne cards, responding to emails, compiling code from earlier in the day and finishing up my essay on Kantian ethics. It’s been a long day, but it doesn’t really explain my slight sense of frustration. Partially that’s because I’m still sick and part of it is having my day dominated by schoolr requirements.

I actually feel quite a bit better now after saying that. I’m just going to do a few more drawings and head to bed. I think I’m going to find a chair to draw…we’ll see how it turns out.