comment on this article

Ten ways to make the debugging process more difficult

Debugging embedded systems can be tough, even with well designed hardware and software. Here are 10 ways to make the process more difficult, guaranteeing you will be the 'hero' when your system ships.

1. Don't design in debug ports
It is useful to have a Jtag port, a spare uart or maybe even an Ethernet interface on the development version of the hardware, but these are luxuries; it is much better to use guesswork and inspiration. Plus, you will have to remove those debug interfaces from the production hardware.

2. Mix logging and tracing techniques
By using printf() liberally, you can affect the time behaviour of your system and hide bugs that will only cause problems when printf() statements are removed. Stack overflows will add to the fun of debugging by causing unpredictable behaviour.
Without a logging mechanism that can be enabled or disabled selectively, you will have more irrelevant output to look at, which makes finding what you are looking for more challenging.

3. Don't use off the shelf debugging tools
Good development tools make the job too easy, they bloat the code, increase execution time by as much as 2% and it takes time to learn how to use them; time better spent sticking to printf() and guesswork.

4. Don't use hardware based debugging tools
If you are tracking down a software problem, you don't want to connect hardware based debugging tools to your target. That Jtag probe is bound to be hard to use. Don't even think about using a trace box, oscilloscope or logic analyser; you might have to ask a hardware engineer for help and that is an admission of defeat.

5. Ignore your configuration management system
Real programmers don't need a configuration management system.
If you have one, the last thing you want to do is use it to help you during debugging – the ability to revert to previous, working, versions of the code and find the specific change that caused a problem makes the process easier than it has any right to be.
Don't test changes one at a time, reverting to the baseline version after each test. Make several changes at once, then test them together. This will be the quickest way to solve the problem and it doesn't matter which change fixed it; all changes have improved the software.

6. Assume there is a bug in the compiler
Mostly, this is not the case. However, it is a good reason not to look more carefully in the right places for the cause of a problem.

7. Don't keep notes
Keeping notes will just slow the process of trying new things and stop you being 'in the zone'. People will say they find it helpful to review their notes to see whether they have overlooked anything, but they are kidding themselves; you really have to be working with your head in the code to make progress.

8. Ignore 'one off' behaviour
If a problem cannot be reproduced, assume it no longer exists. If your system behaves oddly during one test, but not when you run the same test again, you must have imagined it. Of course, there could be marginal timing on a hardware interface, but if it only happened once, don't worry about it.

9. Never ask for help
You know what is really going on. Having to explain the problem to someone else will be a waste of time; it will also mean you have to think about your assumptions. This is one of those times when you don't need a second opinion, especially if it sends you along a path you have not considered.

10. Make last minute changes
It is always a good idea to make one or two last minute changes before software is released; change the compiler's optimisation level or remove your tracing/logging code. Even if these changes unearth some race conditions, they will only cause problems once in a few thousand times.

But seriously …
This list of 'tips', created for fun, is based on experience. Debugging an embedded system is a serious business – it is usually the most stressful part of the development process.

As engineers, we are often under commercial pressure and working to tight deadlines. A lot can rest on our shoulders and being able to think clearly is a luxury that is rarely available.

Author profile:
Ian Willats is managing director of Pebble Bay Consulting.

Author
Ian Willats

Related Downloads
28146\P33.pdf

Comment on this article


This material is protected by MA Business copyright See Terms and Conditions. One-off usage is permitted but bulk copying is not. For multiple copies contact the sales team.

What you think about this article:


Add your comments

Name
 
Email
 
Comments
 

Your comments/feedback may be edited prior to publishing. Not all entries will be published.
Please view our Terms and Conditions before leaving a comment.

Related Articles

Change based testing

A major cause of software bugs is inefficient and incomplete testing. This ...

Life without GitHub?

The software development platform and code sharing repository GitHub celebrates ...

Keeping standards high

Engineers – at least those who are worth their salt – ask questions. And those ...