CONTENTS

Contents

Foreword by Dave Moore, Director of Development, Microsoft

Preface

Acknowledgments

Introduction

Two Critical Questions

What Are Those Gobbledygook Names?

Just What is a Bug?

Chapter 1: Automatic Bug Detection

Watch Your Language

lint—It’s Not That Bad

But My Changes Were Trivial

No More Egg Rolls

Chapter 2: Assert Yourself

A Tale of Two Versions

Introducing Assert

“Undefined” Means “Steer Clear”

The Code That Cried “Danger”

Not For Error-Handling

Are You Making Assumptions Again?

Can the Impossible Happen?

The Silent Treatment

Two Algorithms Are Better Than One

Stop Bugs at the Starting Line

Assertions Are Forever

Chapter 3: Fortify Your Subsystems

Subsystem Checks

Shred Your Garbage

Keep A Journal to Jog Your Memory

Don’t Wait for Bugs to Call

It’s Obvious Once You See It

There’s No Need To Know

Not All Designs Are Equal

A Paradigm Shift

Find Bugs Now, or Find Them Later

Chapter 4: Step Through Your Code

Gain Confidence in Your Code

Forcing Errors in the Code

Data Flow, the Lifeblood of Code

Are You Missing Something?

Try It—You’ll Like It

Chapter 5: Candy-Machine Interfaces

getchar Gets An int, Of Course

Just a Little Extra Thought

The One-Function Memory Manager

Wishy-Washy Inputs

Don’t Fail Me Now

Read Between the Lines

Warn People of the Hazards

The Devil Is in the Details

Chapter 6: Risky Business

Doth Your Data Run Over?

Close Counts Only in Horseshoes

Functions Just “Doing Their Thing”

No ifs, &&s, or Buts

The ?: operator is an if statement too

Rid Your Code of Redundancy

High Risk, No Return

Inconsistency, the Gremlin of Code

Don’t Associate with Failures

A Slap on the Risk

Chapter 7: Treacheries of the Trade

The Need for Speed

A Thief With a Key Is Still a Thief

Take Only What You Need

Keep Private Things to Yourself

Functional leeches

The Old Paint-Stirrer Trick

APL Syndrome

No Hoity-Toity Programming

Waste That Bag of Tricks

Chapter 8: The Rest is Attitude

For my next trick, disappearing bugs

A Fix in Time Saves Nine

Bug-Doctor to the Rescue!

Are You A Code Meddler?

Put “Cool” Features Into Cold Storage

No Free Lunches

Flexibility Breeds Bugs

Ported Code is New Code

“Try” Is A Four-Letter Word

The Sacred Schedule

What’s in a Name?

Testers Wear White Hats

What Are Your Coding Priorities?

You Don’t Get What You Don’t Ask For

Epilogue: Where Do You Go From Here?

Appendix A: Coding Checklists

Design

Implementation

Adding Debug Support

Testing

Debugging

Appendix B: Memory Logging Routines

Appendix C: Answers

Appendix D: The Interview Question

References

About the Author

Copyright

Colophon

FOREWORD

I first met Steve Maguire in 1986 when we hired him at Microsoft to work on Microsoft’s Excel project for Steve Jobs’ then new brainchild, the original Macintosh computer. At Microsoft, we immediately embraced the Macintosh computer and were thrilled to be one of the first companies to develop software for this wonderful new graphical computer. We thought the Macintosh was going to be a game-changer, and it proved to be exactly that.

Let me give you a quick history lesson about how this book, Writing Solid Code, came about and my relationship to it…

I was hired by Bill Gates in 1981 to work for Microsoft’s business applications group. Back then, that meant 7 total programmers working on one business application—Microsoft Multiplan, an MS-DOS-based spreadsheet program. Another 30 programmers were working on our various language compilers, interpreters and operating systems products. The rest of the 100 or so people in the company were in technical writing, sales, marketing, and administration. This was before anybody really knew who Microsoft was. As a company, we were just breaking into public awareness.

At that time, all 7 Multiplan spreadsheet programmers were crammed into one large room in an office building in Bellevue, Washington, on the east side of Seattle. We weren’t even in the same building with the rest of the developers who were working on MS-DOS and Microsoft Basic. They were two blocks away. But that wasn’t a big problem. We were a small company with a vision of what we wanted to accomplish: a computer on every desktop running Microsoft software.

The system we used to develop Multiplan was pretty sophisticated for PC development in those days. We wrote the core product in C when most code back then was still being written directly in assembly language. We did our editing and compilation on a DEC PDP-11 minicomputer running Unix. The C code was compiled into pseudo-code using a homegrown C compiler and we wrote pseudo-code interpreters for each microprocessor that we were targeting at that time.

In 1981, our primary focus had been on shipping original equipment manufacturer (OEM) products. We would work with an OEM, customizing our products to fit the OEM’s hardware and sales channels. We would then ship the OEM a disk and photo-ready copies of the manual. The OEM would do all of the manufacturing of the product, the sales, and the support.

In 1982, we shifted to a retail emphasis.

The OEM focus had allowed Microsoft to travel light. The OEM had to provide all testing, project management, product manufacturing, product shipping, product support, and sales.

With the switch to a retail focus, we had to develop all of these specialized product development and support functions in-house at Microsoft.

In 1984, the complexity of our products and the high standards involved in creating top-notch retail products led us to start up a quality assurance group which we called the “testing group.”

Part of my job as Director of Development was to inventory and disseminate best practices. Steve has captured and described the best techniques we’ve discovered at Microsoft to help us successfully develop multi-million line coding projects.

My desire is that you and your projects can benefit greatly from the hard-earned findings presented in this book.

David M. Moore

Former Director of Development, Microsoft

Redmond, Washington

PREFACE

When I set out to write the first edition of Writing Solid Code twenty years ago, I had a simple thought in mind: Give programmers proven tools, techniques, and philosophies to help them write rock-solid, bug-free code.

At that time, I thought the book would reach a tiny niche market: Very experienced project leads who were running large teams of programmers. I had no idea that the book would become a runaway best-seller, eventually being translated into more than 16 different languages. I certainly didn’t expect that so many software development companies would make the book required reading for their developers. Nor did I anticipate that Universities around the world would use the book in their computer science courses.

Over the years, programmers have regularly asked me if my views have changed much since the book’s original publication. Not only have my views not changed over all those years, but I’ve embraced the concepts and philosophies expressed in the book’s pages even more staunchly. The reason for my enthusiasm comes from the many readers who’ve taken time to contact me telling me how much they’ve benefited by practicing the philosophies that I outlined all those years ago. I especially value the comments I’ve received from the seasoned programmers who’ve been writing commercial code for decades—they, in particular, have the experience to judge whether the book’s suggestions and guidelines make sense in the “real world.”

That’s not to say that there aren’t experienced programmers who’ve read the first edition and expressed concerns about some of the book’s advice.

To give one example, in Chapter 2, I promote the philosophy of using assertions to alert programmers to situations that should never occur in a bug-free program. Some programmers have read that suggestion to mean that you should use assertions for error-handling and they rightfully disagree with that use of assertions. An aside on page 22 of the first edition explicitly covers this point, but covering this important distinction just once in an aside clearly wasn’t enough. In this edition, I revisit this distinction at every opportunity because I so strongly believe in using assertions to help you write rock-solid, bug-free code.

Many readers have given additional insights on the philosophies presented in the first edition, and I’m thrilled to be able to add their thoughts to what has already been presented.

Other readers have written, “Why didn’t you cover this topic, or that topic? These topics are critical to writing bug-free code.”

So true… You could literally write dozens of books on the topic of writing bug-free code.

You could write an entire book on the importance of doing code reviews, and how to do them effectively. You could write another book on the importance of creating unit tests and regression tests and how to create and build those effectively. You could write yet a third book on the importance of upfront design before you launch into writing code.

I could go on, but I think you get the point. There are so many different angles that you could pursue to writing rock-solid, bug-free code.

When I set out to write the first edition of this book in 1992, I created a list of guidelines to determine whether or not to include a topic or guideline in the book. I wanted the book to focus on concepts and philosophies about developing code that weren’t widely known, weren’t widely practiced, and weren’t widely written about. And although I knew I was going to demonstrate the principles using the C programming language, I didn’t want the guidelines to be unique to the C language. With that in mind, I taped a list of guidelines for Writing Solid Code to the wall next to my computer to remind me of these goals. It’s been 20 years now, so I don’t remember all of the guidelines, but I’ll give you a sampling of guidelines that I do remember…

◆

Don’t include topics that have already been covered thoroughly elsewhere, many times. No rehashing. Stick to unknown concepts.

◆

For inclusion, a guideline must be useable that day, immediately after reading the guideline. No “perfect world” scenarios.

◆

The concept or philosophy should cause experienced programmers to think, “Wow. I never thought about it that way…”

◆

Don’t include any guidelines that only make sense to C programmers. (Keep in mind, this is not a book about programming in C. It is a book of concepts and philosophies that programmers can follow to write solid, bug-free code, regardless of any language they use.)

◆

Stick to real-world examples that you’ve personally witnessed in projects, on multiple occasions. Don’t use contrived examples simply to make a point.

◆

Share stories of your own failures...

I had a long list of guidelines taped to my wall, but as I wrote, I kept reminding myself that the concepts and philosophies in the book had to be something not well-known and not well-documented already; the readers had to be able to immediately use any concept presented in the book; the concept or philosophy had to make seasoned veterans think twice about commonly accepted practices that they regularly use; and the concept or philosophy had to be largely language independent.

Yes, I occasionally violated some of these guidelines. But that’s what guidelines are. Guidelines. Not rules. Guidelines simply help you keep your focus on what you’re trying to do, or to communicate.

For this second edition, I briefly considered updating the code examples to use a more hip language such as C#, Java, or any of the popular web-based languages that have flourished. The reality though is that the C language is the lingua franca for programmers. They may not code in C on a regular basis, but every computer science student has—or should have—learned C in their course of studies. As of this second edition of Writing Solid Code in 2013, C is still the most widely understood programming language and that continues to make it the best language for introducing new concepts and ideas.

If you’re programming in a language other than C, take the philosophies in this book and adapt them to your particular programming language.

ACKNOWLEDGMENTS

I’d like to thank everybody who helped make this book a reality. First I would like to thank Mike Halvorson, my acquisitions editor, for letting me take the project at my own speed and for patiently answering the many questions I had for him. I would especially like to thank Erin O’Connor, my manuscript editor, who gave me early feedback on the chapters, and without whose help this book simply would not exist. Erin also encouraged me to relax into my own style, and it certainly didn’t hurt that she laughed at the text’s little jokes. Jeff Carey gave the ideas and the code a good going over, and Kathleen Atkins made many good suggestions. David Braughler at Braughler Books was also instrumental in helping put this second edition together.

I’d also like to thank my father, Joseph Maguire, who in the mid-1970s introduced me to those first microcomputers: the Altair, the IMSAI, and the Sol-20. He is responsible for getting me hooked on computer programming. Evan Rosen, with whom I worked at Valpar International from 1981 to 1983, was a great influence on me, and his knowledge and insight show up in this book. Paul Davis, with whom I had the pleasure to work with for over a decade on various projects, has also shaped my thinking in significant ways.

I‘d like to thank all the people who took the time to read through draft copies of this book to give me technical feedback: Mark Gerber, Melissa Birch Glerum, Chris Mason, Dave Moore, John Rae-Grant, and Alex Tilles. I’d like to especially thank Eric Schlegel and Paul Davis for not only reviewing draft copies of the book but also giving me early help in hammering out the details.

Steve Maguire

Cincinnati, Ohio

February 21, 2013

INTRODUCTION

Years ago I picked up a copy of TEX: The Program, by Donald Knuth, and what I read in the preface astounded me:

I believe that the final bug in TEX was discovered and removed on November 27, 1985. But if, somehow, an error still lurks in the code, I shall gladly pay a finder’s fee of $20.48 to the first person who discovers it. (This is twice the previous amount, and I plan to double it again in a year; you see, I really am confident!)

I have no idea whether Knuth paid anybody $20.48 or even $40.96; that’s not important. What is important is the confidence Knuth had in the quality of his code.

How many programmers do you know who would seriously claim that their code is totally bug-free? How many would publish such a claim and back it up with a finder’s fee?

Programmers could make such claims if they truly believed that their testing groups had found all their bugs. But that’s the problem. How many times have you seen programmers cross their fingers and say, “I hope the testing group has found all the bugs” right before they upload their code to the app store or ship it to their dealers?

Programmers today aren’t sure their code is bug-free because they’ve relinquished responsibility for thoroughly testing it. It’s not that management ever came out and said, “Don’t worry about testing your code—the testers will do that for you.” It’s more subtle than that. Management expects programmers to test their code, but they expect testers to be more thorough; after all, that’s the testing group’s full-time job.

The purpose of this book is to show how programmers can take back the responsibility for writing bug-free code. That doesn’t necessarily mean writing perfect code the first time—it means creating a product that’s bug-free before it first goes into testing. Some programmers may laugh incredulously at such an idea, but this book demonstrates techniques and provides guidelines that programmers can use to work toward that goal.

TWO CRITICAL QUESTIONS

The most important requirement for writing bug-free code is to become attuned to why bugs come about. All of the techniques and guidelines presented in this book are the result of programmers asking two specific questions for every bug they find in their code:

◆

How could I have automatically detected this bug?

◆

How could I have prevented this bug?

The easy answer to both questions would be “better testing,” but that’s not automatic, nor is it really preventive. Answers like “better testing” are so general they have no muscle—they’re effectively worthless. Good answers to these questions result in specific techniques and coding philosophies that eliminate such bugs in the future.

This book is devoted to techniques, guidelines, and coding philosophies that have been found to reduce or completely eliminate entire classes of bugs. Some of the points in this book smack right up against common coding practices, but before dismissing these points with “everybody breaks that guideline,” or “nobody does that,” stop and think it through for yourself. If “nobody does that,” why not? Make sure the reasons are still valid. Coding practices that made sense when FORTRAN was the hot new language in the 1960s may not make sense now.

That’s not to say that you should blindly follow the guidelines in this book. They aren’t rules. They’re guidelines. Too many programmers have taken the guideline “Don’t use goto statements” as a commandment from God that should never be broken. When asked why they’re so strongly against gotos, they say that using goto statements results in unmaintainable spaghetti code. Experienced programmers often add that goto statements can upset the compiler’s code optimizer. Both points are valid. Yet there are times when the judicious use of a goto can greatly improve the clarity and efficiency of the code. In such cases, clinging to the guideline “Don’t use goto statements” would result in worse code, not better.

The guidelines in this book are no different: They are meant to be followed most of the time, and they’re meant to be broken when you can get better results by breaking them.

In addition to the guidelines and techniques, most of the chapters in this book contain a section at the end called “Things to Think About.” Questions in this section of a chapter explore new areas that haven’t been covered in the earlier parts of the chapter. The questions aren’t exercises—they don’t test your comprehension of the concepts presented in the chapter. I’ve tried to introduce at least one new concept in every question, and I’ve provided a complete set of answers in order to pass on as much information as possible. If you normally skip over exercises when you read, I encourage you to think through the exercises and read the answers in Appendix C so that you won’t miss any of the guidelines or techniques I’ve introduced there.

WHAT ARE THOSE GOBBLEDYGOOK NAMES?

By now you’ve probably flipped through the pages of this book and noticed the many strange-looking variable and function names used in the code. Names like pch and ppb are common.

Although names like pch look funny and are hard to pronounce, they are filled with information—once you understand the “Hungarian” naming convention developed by Charles Simonyi—from Hungary—in the early 1970s. The premise behind the Hungarian naming convention is that conveying information is far more important in naming your variables than being able to stand up and read your code aloud during a program review.

The details of the simplified Hungarian naming convention I use in this book are pretty easy. For variables of each data type in your program, you use an abbreviation for the type as part of the variable name. There’s nothing earth-shattering about that—programmers have long called their characters c or ch, their bytes b, their integers i, and so on. The Hungarian naming convention merely enforces this practice for all data types in a program. For example,

char   ch;       /*  a plain old character  */

byte   b;        /*  a byte, which is an unsigned char  */

flag   f;        /*  flags that are always TRUE or FALSE  */

symbol sym;      /*  some sort of symbol structure  */

The convention doesn’t specify what the data type abbreviations should be—only that they be used consistently throughout the program.

Pointer variables pose an interesting problem in that they must point to something. The Hungarian naming convention handles this by specifying that all pointer variable names start with the letter p followed by the abbreviation for the data type that the pointer references. If you were to declare pointers to the data types we just saw you would have the following names:

char   *pch;     /*  character pointer  */

byte   *pb;      /*  byte pointer  */

flag   *pf;      /*  pointer to a flag. */

symbol *psym;    /*  pointer to a symbol structure  */

Pointers to other pointers aren’t any different from pointers to regular data types—you still attach a p to the front of the data type that’s being pointed to. The name for a pointer to a character pointer would be a p tacked to the front of a pch:

char **ppch;     /*  pointer to a character pointer  */

This piling up of data types makes the Hungarian naming convention difficult to read, but the naming convention allows programmers to append a descriptive word or two—each starting with a capital letter—to the abbreviation for the variable type. Not only does this improve readability, but it also makes it possible to distinguish similarly typed variables from each other. The strcpy function, for example, takes two character pointers as arguments, so one possible prototype for the function would be

char *strcpy(char *pchTo, char *pchFrom);

This brings up another point. Because the purpose of the Hungarian naming convention is to increase comprehension, the convention puts more emphasis on what the data types represent than on how they are actually declared. The two arguments to strcpy are character pointers, but more important, they are pointers to zero-terminated strings. Naming strcpy’s arguments as pchs would be correct, but using something like str would be more meaningful:

char *strcpy(char *strTo, char *strFrom);

str s are still character pointers, but when you see the names, you know they’re special character pointers—they point to strings.

Function and array names follow the same convention—they start with the type returned and are followed by a descriptive tag. In formal Hungarian, function names always start with a capital letter, but in this book I have regularized the convention for consistency; there’s no difference in capitalization conventions among function names, array names, and variable names. If the standard malloc and realloc functions were written using Hungarian-style names, they might be prototyped as

void *pvNewBlock(size_t size);

and

void *pvResizeBlock(void *pv, size_t sizeNew);

One benefit of using Hungarian names is that they make it easy to decipher pointer expressions. For example, you’ll see many pointers to pointers in this book, particularly ppb’s.

*ppb = pbNew;

Although this code may look unreadable at first, once you realize that you can cancel *s and ps in such expressions, you can easily understand what is going on. If you cancel the first * and p in the expression above you get

pb = pbNew;

Since the types match—they’re both pbs—you know that the expression is correct. &s and ->s also cancel with ps. Consider these statements:

pb = &b;

b = psym->bLength;

If you cancel the p and the & in the first statement, you get a byte assigned to a byte, and you know the expression is valid. In the second statement, if you cancel the p with the ->, you also get a byte assigned to a byte: b = sym.bLength. This "type calculus," as it is called, makes it easier to tear apart complex pointer expressions.

Although I’ve left a lot unsaid about the Hungarian naming convention, the basic outline here should be enough for you to follow the code in this book.

JUST WHAT IS A BUG?

Before we move on to Chapter 1’s automatic bug detection techniques, I should explain a little bit about the kind of bug this book is preoccupied with. I know you know what a bug is—I don’t need to define the term “bug” for anybody reading this book. But in this book I make a distinction between two classes of bugs: those you introduce while working on a feature and those that remain in your code after you believe the code is finished.

Many programming projects use source code control systems to simplify program development. A programmer checks out a file he or she needs to modify, much the way you would check out a book at the library. The only difference is that the programmer checks out a copy of the file, not the file itself. This allows the programmer to implement new features without actually touching the master source files. Once the programmer has finished implementing a feature and is sure that the code is free of bugs, the file is checked back in, and the source code control system updates the master files accordingly.

With this arrangement, it really doesn’t matter how many bugs the programmer introduces into the code while implementing new features, provided, of course, that all bugs are fixed before the new code is checked into the master sources.

When I say “bug” in this book, I mean bugs that make it into the master sources where they hurt the product and affect the end-user. I don’t expect programmers to write flawless code every time they sit at the keyboard, but I do believe that it’s possible to keep bugs out of the master sources.

The guidelines and techniques in the following chapters describe how to write such bug-free code.

1

AUTOMATIC BUG

DETECTION

Think about this for a moment: How buggy would your code be if your Integrated Development Environment (your IDE) could pinpoint every problem in your code? I’m not just talking about syntax errors, but about every problem, no matter how obscure.

Suppose you had an off-by-one bug in your code and your IDE could somehow detect it and give you a warning like this one:

->line 23: while (i <= j)

^^

Off-by-one error: This should be '<'

Or what if your IDE could find mistakes in your algorithms:

->line 42: int itoa(int i, char *str)

^^^^

Algorithm error: itoa fails when i is -32768

Or suppose your IDE could tell you when you’re passing bad arguments:

->line 318: strCopy = memcpy(malloc(length), str, length);

^^^^^^

Invalid argument: memcpy fails when malloc returns NULL

OK, so maybe this is a bit farfetched, but if your IDE could automatically alert you to such mistakes, how easy do you think it would be to write bug-free programs? Wouldn’t it be trivial—at least compared to what programmers normally go through?

If you were to aim a spy satellite camera at a typical software house, you’d find programmers hunched over their keyboards tracking down reported bugs. Elsewhere, you might find testers attacking the latest internal release, bombarding it with inputs to catch new bugs. You might even find testers checking to be sure that none of the old bugs have sneaked back into the code. If you think searching for bugs this way takes a lot of effort compared to using a smart IDE to catch errors, you’d be right; it also requires a lot of luck.

Luck?

Yes, luck. When a tester finds a bug, isn’t it because he or she happened to notice that some number was wrong, or that a feature didn’t behave as expected, or that the program crashed? Take another look at the hypothetical IDE errors. Would a tester see the off-by-one bug if the program appeared to work despite the problem? What about the other two errors?

It may sound scary, but testers hurl inputs at programs and hope that lurking bugs will somehow show themselves. “Yeah, but our testers are more sophisticated than that. They use code coverage tools, automated test suites, random monkey programs, display snapshots, and a bunch of other stuff.” That may be true, but look at what those tools do. Coverage analysis tells testers what parts of your program aren’t being tested; the testers use that information to devise new inputs to your program. And the other tools are automated forms of the “pound and observe” strategy.

Don’t misunderstand me, I’m not saying that what testers do is wrong. I’m saying that it’s hard to test a program as a black box because all a tester can do is stuff things into the program and watch what pops out. It’s like trying to determine whether somebody is insane. You ask questions; you listen to answers; and you make a judgment call. In the end, you’re never really sure because you don’t know what’s going on inside the other person’s head. You always wonder, “Did I ask enough questions? Did I ask the right questions?”

Don’t rely on black box testing. Instead, lean on every bug-detection feature of whatever IDE, compiler, interpreter, or other development tool you’re using. Eliminate luck and take every opportunity to catch bugs automatically.

WATCH YOUR LANGUAGE

When was the last time you read an advertisement for a leading word processor? If the folks on Madison Avenue wrote it, it probably sounded something like this: “Whether you’re writing a note to Johnny’s teacher or working on the next Great American Novel, WordSmasher can handle it. Effortlessly. And to catch typing mistakes that creep into your masterpiece, there’s a mind-boggling 233,000-word spelling dictionary—51,000 more words than in the nearest competitor’s. So go online to your app store and pick up a copy. WordSmasher. The most revolutionary writing tool since the ball-point pen.”

As users, we’ve been trained by constant marketing propaganda to believe that the bigger the spelling dictionary, the better. But that isn’t true. You can find the words em, abel, and si in any online dictionary, but do you really want the spelling checker in your operating system to allow those words when the words me, able, and is are so common? If you see the word suing in something I write, the odds are astronomical that I meant to type using. It doesn’t matter that suing is a real word; in my writing it is an error.

Fortunately, high-quality spelling checkers will let you delete troublesome words like em from their dictionaries so that you can flag an otherwise legal word as an error. Smart IDEs are no different—they will let you flag otherwise legal language idioms as errors because the idioms are so often used in a mistaken way. Such an IDE would detect the misplaced semicolon on the while line below:

/*  memcpy -- copy a non-overlapping memory block  */

void *memcpy(void *pvTo, void *pvFrom, size_t size)

{

byte *pbTo   = (byte *)pvTo;

byte *pbFrom = (byte *)pvFrom;

while (size-- > Ø);   /* <= ; bug  */

*pbTo++ = *pbFrom++;

return (pvTo);

}

You can tell from the indentation that the semicolon is a mistake, but what the IDE sees is a while statement with a null body and that’s perfectly legal in the C language. To catch unwanted null statements, modern IDEs often provide an optional warning that, if you use it, will automatically alert you to bugs like this one. And for those times when you intentionally want a null statement, you can silence the IDE’s warning by using the IDE’s suggested workaround—using a constant expression that will be optimized away (such as NULL;), or using an empty block {}, or using some other workaround. I’ve used {} here.

char *strcpy(char *pchTo, char *pchFrom)

{

char *pchStart = pchTo;

while (*pchTo++ = *pchFrom++)

{}

return (pchStart);

}

The result: You get the flexibility that null statements provide, but the IDE flags unintentional null statements as errors, automatically, provided that you have maximum error checking turned on.

Disallowing one type of null statement is not that different from deleting the word zeros from your spelling dictionary because you want to be consistent about using the alternative spelling zeroes.

Another common problem in many languages is the unintentional assignment. C is flexible in that it lets you use assignments anywhere you can write an expression. If you aren’t careful, however, this extra flexibility can trip you up. For instance, look at this common bug:

if (ch = '\t')

ExpandTab();

Although it’s clear from the indentation of the code that the intention is to compare the “tab” character to ch, the code actually assigns the “tab” character to ch…

Modern IDEs for C programmers help them catch this bug by flagging such assignments in if statements, and also in for and while constructs, as well as in && and || expressions. The idea behind this feature is that if a programmer is going to accidentally type = instead of ==, the odds are good that it’s going to be in one of these five situations.

This IDE option doesn’t stop you from making assignments in C, but to circumvent the warning you must explicitly compare the result against another value, usually 0 or the nul character. Going back to the strcpy example we just saw, instead of writing the loop as

while (*pchTo++ = *pchFrom++)

{}

which would generate the optional warning, you would write:

while ((*pchTo++ = *pchFrom++) != '\Ø')

{}

Another class of bugs falls into the category of “argument bugs.” Years ago when I was first learning the C language and its library functions, I used to call fputc this way:

fprintf(stderr, "Unable to open file %s.\n", filename);

⋮

fputc(stderr, '\n');

That might look OK, but the arguments to fputc are in the wrong order. For some reason, I had incorrectly “learned” that the stream pointer (stderr) was always the first argument to any of C’s stream functions. That isn’t true, so I often passed garbage to those routines. Fortunately, a smart IDE will automatically catch such bugs because the arguments don’t match the types in the function’s declaration.

The null statement alert, erroneous assignment warning, and function argument checks are just a few of the warning options found in many IDEs; often there are more. The key point is that using optional IDE warnings will alert you to possible bugs automatically, much the way the spelling checker alerts you to possible misspellings.

It’s sort of like the sentence “I moved into a new home and I have 3 unpacked boxes in the kitchen.” The question though, is whether those 3 boxes are unpacked—meaning empty—or have they yet to be unpacked and are full? While the sentence is technically correct English, the meaning is not clear and should be restated to more accurately express what’s meant. I’ve asked this question to groups of people at parties and you’d be surprised by the debates that can get started. “Absolutely, the boxes are empty.” “What? You’re crazy, those boxes are full.” Fun! Use caution…

But I digress…

My point is that your code can be technically valid, but not do what you intend it to do. That’s why you need to use every tool at your disposal to detect bugs and probable bugs automatically, at the earliest time in the development process—when you first write the code.

Peter Lynch, arguably the best mutual fund manager of the 1980s, once said that the difference between investors and gamblers is that investors take every opportunity, no matter how small, to tilt the advantage their way; gamblers, in his view, rely on luck. Apply that concept to your programming and enable every optional IDE warning; view the warnings as a risk-free, high-return investment in your program. Don’t ask, “Should I enable this warning?” Instead ask, “Why shouldn’t I enable it?” Turn on every warning unless you have an excellent reason not to.

Enable all optional IDE warnings.

lint—IT’S NOT THAT BAD

If for some reason you’re not able to use an advanced IDE, or the IDE you do use doesn’t automatically alert you to technically correct, but error-prone code, you can still incorporate such technology into your development process.

For example, if you’re a C developer, there’s been a tool available for decades called lint. You simply run lint on your source files and it will alert you to every known problematic issue in your code even if your C code is technically correct. You can think of lint as sort of a C grammar checker for your code. Do you have dangling participles? Have you ended a sentence with a preposition? You get the idea.

lint is very picky; so picky in fact that many C programmers refuse to use it even though this tool is quite extraordinary at detecting bugs automatically.

To use lint effectively, C programmers must learn to use the more explicit workarounds such as those that I’ve covered in this chapter. After they learn these subtleties in writing more explicit C code, they can program all day long and lint won’t utter a peep.

Whether you choose to use lint or not comes back to your philosophies about writing code—is your mindset to write rock-solid code the first time, or is your mindset to avoid some hassle generated from a picky tool? What’s your intended outcome?

Historically, C’s lint program has been so valuable in detecting bugs automatically that almost every modern computer language today has some form of lint of its own. Just search online for “lint for language” and you’ll find valuable tools that can help you write rock-solid code.

For the curious, smart IDE’s that do check for problematic code simply run lint in the background as you type code and they report lint’s findings. No magic happening here. That’s why the IDE for the language you program in probably has lint-type code checking “built in” to it.

Use lint to automatically alert you to probable bugs that your IDE may miss.

BUT MY CHANGES WERE TRIVIAL

I was having lunch with one of the technical reviewers for this book and he asked me if I was going to include a section on unit tests. I told him No because while unit tests are instrumental in writing bug-free code, they fall into a different category than this book’s focus. I explained that this book’s focus was on the philosophies that programmers can use to write rock-solid bug-free code on a daily basis.

He said, “No, that’s not what I meant. I meant are you going to point out that programmers should actually run the project’s unit tests before merging their changes into the master sources?”

He went on, “One of the programmers on our team just introduced a serious bug into our master sources because he didn’t run the unit tests after making his changes.”

That was surprising for me to hear because Microsoft requires programmers to run all unit tests before merging their changes into the master source files.

“Did you ask him why he didn’t run the tests?”

“He said that he didn’t write any new code—he just moved some existing code around. He didn’t think he needed to run the tests because his changes were so trivial.”

The story reminded me of a programmer who once didn’t even try to compile one of his changes before merging the code into the master sources. I found out about it, of course, because I couldn’t compile the project without getting an error. When I asked the programmer how he could miss a compiler error, he said, “The change I made was so trivial, I didn’t think I needed to.”

Neither of these bugs should have made it into the master sources because both could have been caught automatically, with almost no effort.

So why do programmers make such mistakes?

Mainly it’s because they get overconfident of their ability to write correct code and don’t feel the need to check that their code works as advertised.

Sometimes it may seem that you can skip steps designed to keep bugs out of your code, but any time you take a shortcut, you’re asking for trouble. I doubt that there are many programmers who would “finish” a feature without even compiling the code—I know of just that one incident—but the temptation to bypass time-consuming unit tests is stronger, especially for simple changes that you make to the code.

If you find yourself about to bypass a step that could easily detect bugs for you, stop yourself and instead make use of every tool at your disposal. Unit tests are meant to help you automatically catch bugs, but they can’t do their job if you don’t run them.

If you have unit tests, use them.

NO MORE EGG ROLLS

How many programmers do you know who prefer to spend their time tracking down and fixing bugs instead of writing new code? I’m sure there are such programmers, but I’ve never met one. The programmers I know would give up takeout Chinese food for life if you promised them that they’d never have to track down another bug.

As you write code, take advantage of every opportunity to catch bugs automatically with little effort. Think about compiler errors, IDE warnings, lint warnings, and unit test failures. How much skill does it take to find those errors? Almost none. How many bugs would make it into your product if none of the bugs required much skill or effort to detect?

If you want to find bugs quickly and easily, use those features of your tools that tell you where the bugs are. The sooner you know where the bugs are, the sooner you can fix them and move on to more interesting work.

QUICK REVIEW

◆

The best way to eliminate bugs in your code is to find them as early and as easily as possible. Look for ways to catch bugs automatically, with minimal effort.

◆

Strive to reduce the amount of programmer skill necessary to catch bugs. Optional IDE warnings or lint warnings don’t require any programmer skills to detect bugs.

THINGS TO THINK ABOUT

1.

Suppose you’re using your IDE or lint option to disable assignments in while conditions. Why would that catch the precedence bug in the code below?

while (ch=getchar() != EOF)

⋮

2.

You saw how you could use your IDE or lint to catch unintentional null and assignment statements. Suggest ways that an IDE for C programmers could optionally warn of the common problems below. How would you bypass the warnings?

a.

if (flight == 063) where you think you’re testing for Flight 63 when in fact, because the leading 0 forces 063 to be interpreted as an octal constant in C, C++, Objective C, and other C-based offshoots, that you’re actually testing for flight 51.

b.

if (pb != NULL &*pb != 0xff) where you accidentally typed & instead of &&, causing *pb!= 0xff to be executed even if pb is NULL.

c.

quot=numer/*pdenom; where in spite of your intentions, the /* is interpreted as the start of a comment.

d.

word = bHigh<<8 + bLow; which, because of precedence rules in C, is interpreted as word = bHigh<<(8 + bLow); despite your intentions.

3.

How could your IDE or lint automatically alert you to possible “dangling-else” bugs? How would you to silence the warning?

4.

Take another look at this coding error:

if (ch = '\t')

ExpandTab();

Instead of disabling simple assignments in if statements, you could use another popular way of automatically catching such bugs. Simply develop the coding habit of putting the constant on the left-side of the comparison:

if ('\t' = ch)

ExpandTab();

This way if you type = instead of ==, your IDE or lint will squawk because you can’t assign something to a constant. How thorough is this solution? Why is this approach not as automatic as your IDE warnings or lint settings?

2

ASSERT YOURSELF

Using your IDE to automatically detect bugs is great, but I’d bet that if you were to review the outstanding bugs in your project, you’d find that your IDE would catch just a small percentage of them. What’s more, I’d bet that if you were to isolate each bug, you’d find that the code would work correctly most of the time.

Take a look at this simple piece of code:

strCopy = memcpy(malloc(length), str, length);

This code will work in every case except the one in which malloc fails. When that happens, malloc will pass a NULL pointer to memcpy, and memcpy can’t handle that. If you’re lucky, your code will crash and you’ll discover the bug long before you ship; if you’re not lucky, your customers will bear the brunt of the bug and blog about it on every social network they are part of.

In this example, your IDE can’t automatically detect this particular bug or any bug like it. Nor can it catch bugs in your algorithms, verify your assumptions, or in general check the validity of data being passed around.

Finding these kinds bugs is hard. It takes a skilled programmer, or skilled tester, to consistently root them out. But finding these kinds of bugs automatically is easy, if you know how.

In this chapter, I’ll be talking about C’s assert mechanism to automatically detect bugs in C code. And like lint from the last chapter, the assert mechanism is so powerful that every modern language supports it. If you’re not a C programmer, simply do an online search for “assert language” to find the details for your language. The concept is universal, so read on…

A TALE OF TWO VERSIONS

Let’s jump right in and see how you could automatically catch the memcpy bug above. The easiest solution is to have memcpy check for NULL pointers and abort with an error message if it finds one. Here’s how:

/*  memcpy -- copy a non-overlapping memory block  */

void *memcpy(void *pvTo, void *pvFrom, size_t size)

{

byte *pbTo   = (byte *)pvTo;

byte *pbFrom = (byte *)pvFrom;

if (pvTo == NULL  ||  pvFrom == NULL)

{

fprintf(stderr, "Bad args in memcpy\n");

abort();

}

while (size-- > Ø)

*pbTo++ = *pbFrom++;

return (pvTo);

}

Nobody is going to slip a NULL pointer past this function. The only problem is that the test doubles the size of the code and slows it down. If you’re thinking this is a case in which the cure is worse than the disease, I think you’re right; the tests aren’t practical.

But what if you kept two versions of your program? One fast and sleek that you ship, and the other slow and fat because it contains the extra checks. You would maintain both versions in the same sources and for C programmers, you would use the C preprocessor to conditionally include or exclude the checks.

For example, you might compile the NULL pointer tests only when DEBUG is defined:

void *memcpy(void *pvTo, void *pvFrom, size_t size)

{

byte *pbTo   = (byte *)pvTo;

byte *pbFrom = (byte *)pvFrom;

#ifdef DEBUG

if (pvTo == NULL  ||  pvFrom == NULL)

{

fprintf(stderr, "Bad args in memcpy\n");

abort();

}

#endif

while (size-- > Ø)

*pbTo++ = *pbFrom++;

return (pvTo);

}

The idea is to maintain both debug and non-debug (that is, ship) versions of your program. While writing code, you compile the debug version and use it to catch bugs automatically as you add features. Later, when you’ve finished, you compile a ship version free of DEBUG code and make that version available to your end-users.

Of course, you wouldn’t really want to wait until the last minute to run the code you intend to ship—that wouldn’t be smart. But throughout development, you should exercise the debug version, mainly because, as we’ll see in this chapter and the next, running the debug version can drastically reduce the time required to develop bug-free code. Imagine how robust your application would be if every function did some minimal error checking, testing for conditions that should never happen.

The trick, of course, is to ensure that debug code is strictly extra code that isn’t necessary in the final product. There are some gotcha’s, but I’ll cover those as they arise.

The easiest way to think of debug code is to imagine that you and your team have just finished your project, and you’re about to release it to the public. In your mind, you’re done. You’re ready to Rock and Roll.

But, at the last moment, you think, “Hmmm… Just to be sure there aren’t any lurking bugs, let’s add some extra debug code to the project to verify that all of the assumptions we’ve made are valid.” None of this extra debug code is in any way necessary to your final project; it just alerts you to points in the code where assumptions fail.

This distinction is important: debug code is there only to alert you to bugs in the code. Debug code has nothing to do with error handling, which you must do in the code that you release.

Maintain both ship and debug versions of your program.

INTRODUCING ASSERT

Let’s be honest here. The debug code I put in memcpy looks awful and overwhelms the function. I don’t know many programmers who would stand still for that, even if it were for a good cause. That’s why some clever programmer back in the early 1970s decided to hide all that debug code in a macro named assert which is now standard and defined in C’s assert.h header file.

assert is nothing more than a repackaged form of the #ifdef code we saw before, but when you use the macro, it takes one line instead of nine:

void *memcpy(void *pvTo, void *pvFrom, size_t size)

{

byte *pbTo   = (byte *)pvTo;

byte *pbFrom = (byte *)pvFrom;

assert(pvTo != NULL  &&  pvFrom != NULL);

while (size-- > Ø)

*pbTo++ = *pbFrom++;

return (pvTo);

}

assert is a debug-only macro that aborts execution if its argument is false. You can see in this code example that if either pointer is null, the assert will fire.

assert is not a macro you just throw together; you must define it carefully so that it won’t cause important differences between the ship and debug versions of your program. assert should not disturb memory, initialize data that would otherwise be uninitialized, or cause any other side effects. You want your debug program to behave exactly like the ship version. That’s why assert is a macro and not a function; if it were a function, calling it could cause unexpected memory or code swapping. Remember, the programmers who use assert view it as a harmless test that they can safely use no matter what state the system is in.

You should also be aware that once programmers learn to use assertions, they often redefine the assert macro to be more useful or to work better with their particular development system. For instance, instead of having assert abort execution when an error occurs, programmers sometimes redefine assert so that the macro hops into a debugger at the point of the error. Some versions of assert even give you the choice of continuing the program’s execution as though the failure never happened.

If you decide to define your own version of the assertion macro, consider using a name other than assert so that you leave the standard macro untouched. In this book, I’ll be using a nonstandard assertion macro, so I’ve given it the name ASSERT so that it stands out in code. The major difference between the assert and ASSERT macros is that assert is defined as an expression that you can use freely anywhere in your code, but ASSERT is a statement, which restricts its use. With assert you can write

if (assert(p != NULL), p->foo != bar)

⋮

but if you try that with ASSERT, you’ll get a syntax error. That’s intentional. Unless you plan to use assertions in expression contexts, you should define ASSERT as a statement so that the compiler will generate an error if you mistakenly use it in an expression. Remember, every bit helps in the effort to root out bugs. Why allow flexibility that you don’t use?

Here is one way you could define the ASSERT macro using C’s preprocessor:

#ifdef DEBUG

void _Assert(char *, unsigned);   /*  prototype  */

#define ASSERT(f) if (f) {} else _Assert(__FILE__, __LINE__)

#else

#define ASSERT(f)

#endif

You can see that if DEBUG is defined, ASSERT will expand to an if statement. The empty block {} in the if may seem strange, but you need both the if and the else statements to prevent unexpected dangling-if problems. And you might think that you need a final semicolon after the closing ) in the call to _Assert, but you don’t, because you provide the final semicolon when you use ASSERT:

ASSERT(pvTo != NULL  &&  pvFrom != NULL);

When ASSERT fails, it calls _Assert with the file name and line number provided by the preprocessor through the __FILE__ and __LINE__ macros. _Assert prints an error message to stderr and then aborts:

void _Assert(char *strFile, unsigned uLine)

{

fflush(stdout);

fprintf(stderr, "\nAssertion failed: %s, line %u\n",

strFile, uLine);

fflush(stderr);

abort();

}

In C, you need the calls to fflush to write out any buffered output before you execute abort. You also want to fflush stdout before flushing stderr in case they both point to the same device. This ensures that fprintf will display the error message only after all other output to stdout.

Now, if you were to call memcpy with a NULL pointer, ASSERT would catch the bug and display something like:

Assertion failed: string.c, line 153

Regardless of how you ultimately define your version of ASSERT , use it to validate the arguments passed to your functions. If you check data at every entry point, you’ll catch bugs automatically, as they occur.

Use assertions to validate function arguments.

“UNDEFINED” MEANS “STEER CLEAR”

If you were to stop and read the ANSI C definition for the memcpy routine, you would see that the very last line reads, “If copying takes place between objects that overlap, the behavior is undefined.” If you rely on memcpy to behave in a particular way when you call it with overlapping blocks, you’re making an assumption about behavior that can vary from one compiler to the next, or even between releases of the same compiler.

I’m sure there are programmers who deliberately exploit undefined behavior, but I think most programmers intelligently avoid doing this. Those who don’t should learn to. Most programmers view undefined behavior as illegal behavior, and that’s where assertions come in handy. If you were to call memcpy with an overlapping block, wouldn’t you want to know about it?

You can beef up memcpy by adding an assertion to verify that the two blocks never overlap:

/*  memcpy -- copy a non-overlapping memory block  */

void *memcpy(void *pvTo, void *pvFrom, size_t size)

{

byte *pbTo   = (byte *)pvTo;

byte *pbFrom = (byte *)pvFrom;

ASSERT(pvTo != NULL  &&  pvFrom != NULL);

ASSERT(pbTo >= pbFrom+size  ||  pbFrom >= pbTo+size);

while (size-- > Ø)

*pbTo++ = *pbFrom++;

return (pvTo);

}

How that one-line overlap check works may not be obvious, but it’s easy to figure out if you think of the two blocks of memory as cars in line at a stop light. There you know that the cars don’t overlap if the back bumper of one car is in front of the front bumper of the other car. The check implements that idea: pbTo and pbFrom are the back bumpers of the two blocks, and pbTo+size and pbFrom+size are the spots just in front of the front bumpers of the blocks. That’s all there is to it.

And by the way, if you don’t see why all this overlapping stuff is important, just think about the case in which pbTo is equal to pbFrom+1 and you move at least 2 bytes—memcpy will trash the memory as it’s moving it.

Notice that if the overlap assertion fires in this example, there are two possible bugs that are exposed. Either the calling code intended to copy overlapping blocks and should be calling C’s memmove function instead of memcpy, or the calling code never intended to copy overlapping blocks which means you have to look into why an overlapping copy occurred. In rock-solid code neither of these cases should ever occur.

Are you beginning to see how such checks sprinkled throughout your code can help you detect bugs automatically in a controlled fashion?

Dealing with undefined behavior is particularly important if you provide code libraries (or operating systems) to other programmers. If you have ever developed such libraries, you know that other programmers will make use of all sorts of undefined behavior as they “try things” to get the result they want. The consequences really show up with you release a new and improved library. Invariably, you find that while your library is 100 percent compatible with the last version, half the applications have problems when they try to use it. The reason: The new library is not 100 percent compatible with the old “undefined behavior.”

In the future, stop and review your code for undefined behavior. If you find undefined behavior, either remove it from your design or include assertions to notify programmers when they’ve crossed over to the dark side.

Strip undefined behavior from your designs, or use assertions to catch illegal uses of undefined behavior.

THE CODE THAT CRIED “DANGER”

While we’re on the subject, I’d like to talk more about that memcpy overlap assertion for a moment. Here it is again:

ASSERT(pbTo >= pbFrom+size  ||  pbFrom >= pbTo+size);

Suppose you called memcpy and the assertion above failed. When you looked it up, would you know what was wrong if you’d never seen an overlap check before? I know I probably wouldn’t. But that’s not to say that the code is tricky or unclear—it is, after all, a straightforward overlap check. But being straightforward and being obvious are not the same thing.

Take my word for it, there are few things more frustrating than to track an assertion failure to somebody else’s code and then have no idea what the assertion means. Instead of fixing the problem, you waste time just trying to figure out what the problem is.

That’s not all.

Programmers do, on occasion, write buggy assertions, but it’s hard to tell whether you should fix the program or to fix the assertion if you can’t figure out what the assertion is checking for.

Fortunately, it’s easy to solve this problem—just add comments to assertions that aren’t readily obvious. I know this sounds basic, but it’s amazing how rarely programmers do this. They go to all the trouble to protect you from danger, but then they don’t tell you what the danger is.

It’s as if you were walking through the woods and saw a big red DANGER sign nailed to a tree. But what danger? Falling trees? Abandoned mine shafts? Bigfoot? Unless you tell people what the danger is (or unless it’s obvious), you’re not helping them. People in the woods will ignore the sign. Similarly, programmers will ignore any assertions they don’t understand—they’ll assume the assertions are wrong and rip them out.

Even better, if the assertion has a probable solution, note that.

When a programmer calls memcpy with overlapping blocks, there’s a good chance that this is exactly what he or she wants to do, but is unaware of the overlap restriction in memcpy’s implementation.

A strategic comment in memcpy can point out that the slower, but more flexible, memmove should be used for overlapping blocks:

/*  Blocks overlap? Use memmove instead. */

ASSERT(pbTo >= pbFrom+size  ||  pbFrom >= pbTo+size);

You don’t need to write tomes. One approach is to use a short, well-thought-out question. This can be far more informative than an entire paragraph that methodically explains every detail. But be careful. Don’t suggest a solution in your comments unless you’re certain that those comments will guide other programmers in the right direction.

Don’t waste people’s time.

Document unclear assertions.

NOT FOR ERROR-HANDLING

When programmers start using assertions, they sometimes use them incorrectly to test for real errors that they must handle, not illegal conditions that should never occur in bug-free code. For example, look at the assertions in this strdup function:

/*  strdup -- allocate a duplicate of a string  */

char *strdup(char *str)

{

char *strNew;

ASSERT(str != NULL);    /* str must point to something */

strNew = (char *)malloc(strlen(str)+1);

ASSERT(strNew != NULL);   /* Check for failure */

strcpy(strNew, str);

return (strNew);

}

In this code, the first assertion is a correct use because it tests for an illegal condition that should never occur if the program is working correctly. The second assertion is quite different—it is testing for an error condition. The call to malloc can definitely fail, although perhaps very rarely, but it can fail and that failure must be handled correctly in normal code:

/*  strdup -- allocate a duplicate of a string  */

char *strdup(char *str)

{

char *strNew;

ASSERT(str != NULL);    /* str must point to something */

strNew = (char *)malloc(strlen(str)+1);

if (strNew != NULL)

strcpy(strNew, str);

return (strNew);

}

Let’s take a look at another example of an incorrect use of ASSERT using C’s getline call which reads a line of text from a file or terminal:

/*  getline -- read a \n delimited line into a buffer. */

void getline(char *pch)

{

int ch;    /* ch *must* be an int */

do

ASSERT((ch = getchar()) != EOF);

while ((*pch++ = ch) != '\n');

}

The call to getchar can definitely fail, returning the EOF value. This is not a bug in the code, it’s simply a resource allocation failure. You cannot use ASSERT for a real runtime error. You must actually write code to handle that error.

I’m harping on this point because programmers first learning the assertion concept often confuse using assertions with error-handling. So let me state this one more time… If a real runtime error can happen, you must write code to handle that error. If, on the other hand, you notice something that simply can’t happen, ever, in bug-free code, then use an assertion to verify that assumption.

Assertions and DEBUG code are for tests that reveal bugs, never for error-handling.

ARE YOU MAKING ASSUMPTIONS AGAIN?

Sometimes when you write code, you need to make assumptions about the target environment, but not always. For example, the memset routine below makes no assumptions about the target environment and should work with any ANSI C compiler:

/*  memset -- fill memory with a "byte" value  */

void *memset(void *pv, byte b, size_t size)

{

byte *pb = (byte *)pv;

while (size-- > Ø)

*pb++ = b;

return (pv);

}

But for many environments, you can write a faster memset routine by packing a larger data type with the fill value and then using the packed value to fill memory using fewer stores. For example, on a processor that packs 4 bytes into a long, the memset routine below could fill up to four times faster than the portable version above.

/*  longfill -- fill memory with a "long" value. Returns a

*  pointer to the first long value after the last long

*  value filled.

*/

long *longfill(long *pl, long l, size_t size);

void *memset(void *pv, byte b, size_t size)

{

byte *pb = (byte *)pv;

if (size >= sizeThreshold)

{

unsigned long l;

l = (b << 8)  | b;    /*  Pack a long with 4 bytes  */

l = (l << 16) | l;

pb = (byte *)longfill((long *)pb, l, size / 4);

size = size % 4;

}

while (size-- > Ø)

*pb++ = b;

return (pv);

}

This code is fairly straightforward, except possibly for the test against sizeThreshold. If it’s not obvious why this test is desirable, consider that it takes time to pack a long with four copies of a byte. There is also some overhead in calling the longfill function. The test against sizeThreshold ensures that memset won’t fill using longs unless doing so would be faster than not doing so, even accounting for the extra overhead.

The only problem with this new version of memset is that it makes a number of assumptions about the compiler and the operating system. The code explicitly assumes that longs use 4 bytes of memory and that bytes are 8 bits wide. These assumptions are still true for many processors, but have changed for more powerful processors. Still, that doesn’t mean you should blithely let the code operate on its assumptions, because if there’s one thing you can count on, it’s that what is true today will probably not be true a decade from now.

Some programmers would “improve” the routine by writing it so that it’s more portable:

void *memset(void *pv, byte b, size_t size)

{

byte *pb = (byte *)pv;

if (size >= sizeThreshold)

{

unsigned long l;

size_t sizeSize;

l = Ø;

for (sizeSize = sizeof(long); sizeSize-- > Ø; NULL)

l = (l << CHAR_BIT) | b;

pb = (byte *)longfill((long *)pb, l, size / sizeof(long));

size = size % sizeof(long);

}

while (size-- > Ø)

*pb++ = b;

return (pv);

}

This code may look more portable since it makes heavy use of the sizeof operator, but looks don’t mean anything. You would still have to review the code if you need to use it on a processor that requires storing long values on an even address as many processors have historically required. On these processors, if you try and store long values at an odd address, your code will crash and burn…

So, what should you do?

In this case, you simply shouldn’t try to write memset as a portable function, but should instead accept that it is non-portable and protect yourself against change. If you had to implement a faster memset for a processor that requires long values to be stored at an even address, you could avoid the odd-aligned problem by byte-filling until you’re even-aligned, and only then switch to long-filling.

And although being even-aligned would work correctly for these processors, you would get even better performance if these 4-byte long-fills were aligned on 4-byte boundaries. As for the other assumptions, you could verify those using assertions and conditional compilation:

void *memset(void *pv, byte b, size_t size)

{

byte *pb = (byte *)pv;

#ifdef FourByteLongs

if (size >= sizeThreshold)

{

unsigned long l;

ASSERT(sizeof(long) == 4  &&  CHAR_BIT == 8);

ASSERT(sizeThreshold >= 3);

/*  byte-fill until long aligned. */

while (((unsigned long)pb & 3) != Ø)

{

*pb++ = b;

size--;

}

/*  Now pack a long and long-fill the rest. */

l = (b << 8)  | b;

l = (l << 16) | l;

pb = (byte *)longfill((long *)pb, l, size / sizeof(long));

size = size % sizeof(long);

}

#endif /*  FourByteLongs  */

while (size-- > Ø)

*pb++ = b;

return (pv);

}

As you can see, I’ve bracketed the machine-specific code with the FourByteLongs preprocessor definition. Not only will this preprocessor definition keep the non-portable code from being accidentally used on a different target, but by searching for every occurrence of FourByteLongs, you can isolate all target-specific code.

I’ve also added a straightforward assertion to verify that longs use 4 bytes of memory, and that bytes are 8‑bits wide. If these assertions ever fire, you’re automatically alerted that something is wrong.

With these changes, the code is explicitly marked non-portable, and all the assumptions have been eliminated or verified with an assertion. These measures make the function much less likely to be used incorrectly.

Either remove implicit assumptions, or assert that they are valid.

CAN THE IMPOSSIBLE HAPPEN?

The inputs to a function don’t always come in as formal parameters. Sometimes you get only a pointer to the inputs. Take a look at this simple decompression routine:

byte *pbExpand(byte *pbFrom, byte *pbTo, size_t sizeFrom)

{

byte   b, *pbEnd;

size_t size;

pbEnd = pbFrom+sizeFrom;  /* Point just after buffer. */

while (pbFrom < pbEnd)

{

b = *pbFrom++;

if (b == bRepeatCode)

{

/*  Store "size" copies of "b" at pbTo. */

b = *pbFrom++;

size = (size_t)*pbFrom++;

while (size-- > Ø)

*pbTo++ = b;

}

else

*pbTo++ = b;

}

return (pbTo);

}

This code copies one data buffer to another, but in the process it looks for packets of compressed characters. If it finds the special byte bRepeatCode in the data, it knows that the next 2 bytes are a character to repeat and the number of times to repeat it. Although it’s simplistic, you could use the routine in something like a programmer’s editor, in which text often has many consecutive tab or space characters for indentation.

To make pbExpand more robust, you could assert that pbFrom, pbTo, and sizeFrom are valid on entry, but you can do more than that. You can validate the buffer data as well.

It always takes 3 bytes to encode a run, so the compression routine never packs just two consecutive characters; and while it could pack three, there’s no real benefit in that. It strictly packs runs of four or more characters.

There is one exception. If the original data contains bRepeatCode before being encoded, it has to be specially handled so that when pbExpand comes along later, it won’t go crazy thinking it’s got a compressed packet. When the compression routine finds bRepeatCode in the original data, it bundles it into a packet in which bRepeatCode itself is repeated one time.

In short, for every packet, size must be at least 4, or else the byte must be bRepeatCode and size must be 1. You can use assertions to verify this:

⋮

if (b == bRepeatCode)

{

/*  Store "size" copies of "b" at pbTo. */

b = *pbFrom++;

size = (size_t)*pbFrom++;

ASSERT(size >= 4  ||  (size == 1  &&  b == bRepeatCode));

⋮

If this assertion fails, either pbFrom points to garbage, or the compression routine has a bug in it. In either case, it’s a bug that might not otherwise be obvious.

Use assertions to detect impossible conditions.

THE SILENT TREATMENT

Suppose you were hired to write the software for a nuclear reactor and you had to handle the case in which the core overheats.

Some programmers might attack this case by automatically dumping water into the core, inserting cooling rods, or doing whatever it is you do when you’re trying to cool a reactor. And as long as the program had everything under control, it wouldn’t alert the staff to the problem.

Another programmer might choose to always alert the reactor staff whenever the core overheats. The computer could still automatically take care of things, but the operators would always know about it.

Which way would you implement the code?

I doubt there would be much disagreement on this one; you would alert the operators. That the computer can restore the reactor to normal operation is irrelevant. Cores don’t overheat spontaneously—something unusual has to happen for things to go awry, and when they do, somebody had better figure out what that unusual something is so that it doesn’t happen again.

Surprisingly, programmers, and particularly experienced programmers, write code every day that quietly fixes problems whenever something unexpected happens. They even code that way intentionally. You probably do it yourself. Of course, what I’m talking about is the practice of using defensive programming.

In the last section, I showed you code for pbExpand. That function uses defensive programming. This revised version does not—look at the two while conditions:

byte *pbExpand(byte *pbFrom, byte *pbTo, size_t sizeFrom)

{

byte   b, *pbEnd;

size_t size;

pbEnd = pbFrom+sizeFrom;  /* Point just after buffer. */

while (pbFrom != pbEnd)

{

b = *pbFrom++;

if (b == bRepeatCode)

{

/*  Store "size" copies of "b" at pbTo. */

b = *pbFrom++;

size = (size_t)*pbFrom++;

do

*pbTo++ = b;

while (--size != Ø);

}

else

*pbTo++ = b;

}

return (pbTo);

}

Even though this code more accurately reflects the algorithm, few experienced programmers would actually code the algorithm this way. You’d have a better chance of getting them into a two-person Cessna that had no seat belts and no doors. The code feels too risky.

They’d think, “I know pbFrom should never be greater than pbEnd in that outer loop, but what happens if that ever does happen? Hmm. I’d better make sure the loop shuts down if this impossible case ever comes up.”

They’d use the same logic for the inner loop. Even though size should always be greater than or equal to 1, using a while loop instead of a do loop keeps the code from crashing if size is ever 0 on entry.

It seems reasonable, even smart, to protect yourself from these impossible scenarios. But what if pbFrom somehow bounces past pbEnd? Are you more likely to spot this bug in the risky version we just saw or in the defensive version we saw earlier?

The risky version will probably crash since pbExpand would tromp about decompressing everything in memory. You’re definitely going to notice that. The defensive version, on the other hand, exits before pbExpand can do much, if any, damage. You still might notice the bug, but I wouldn’t bet my money on it.

Defensive programming is often touted as a better coding style, but it hides bugs. Remember, the errors we’re talking about should never happen, and by safely handling them, you make it harder to write bug-free code. This is especially true when you have a bouncing pointer such as pbFrom that gets bumped by different amounts each time through the loop.

Does that mean you should stop programming defensively?

The answer of course is No.

Programming defensively hides bugs, but it does serve a valuable purpose. The worst thing a program can do is crash and lose data that a user might have spent hours creating. And in a less-than-ideal world in which programs do crash, anything you can do to prevent data loss is worthwhile. Defensive programming works toward achieving this goal. Without it, your code would be a house of cards ready to crumble with the slightest change in your hardware or operating system. At the same time, you don’t want to hide bugs by programming defensively.

Suppose pbExpand gets called with invalid arguments. Specifically, suppose that sizeFrom is a bit too small and the last byte of the data buffer happens to be bRepeatCode. Since this will look like a compressed packet, pbExpand will read 2 bytes too many from the data buffer and bump pbFrom beyond pbEnd. The result? The risky version of pbExpand will probably crash, and the defensive one will probably save the user from losing data, although it can still wipe out as much as 255 bytes of unknown data. You want both behaviors, but in different versions of your program. You want the debug version to alert you to the bug, and you want the ship version to recover safely, with no loss of data. The solution is to write your code using defensive programming the way you always have but use an assertion to alert you if things go haywire:

byte *pbExpand(byte *pbFrom, byte *pbTo, size_t sizeFrom)

{

byte   b, *pbEnd;

size_t size;

pbEnd = pbFrom+sizeFrom;   /* Point just after buffer. */

while (pbFrom < pbEnd)

{

b = *pbFrom++;

⋮

}

ASSERT(pbFrom == pbEnd);

return (pbTo);

}

This new assertion simply verifies that the code terminated correctly. In the ship version, the defensive code helps protect the user if anything goes wrong, but in the debug version, the bug is still reported. If that’s not having your espresso and drinking it too, I don’t know what is.

Still, you don’t need to be obsessive about this. If pbFrom were always bumped by 1 each time through the loop, it would take a stray cosmic ray to knock it beyond pbEnd and cause problems. In such cases, assertions don’t buy you anything. Look at your code and use your common sense.

One last point. Loops are only one area in which programmers routinely program defensively. No matter where you employ defensive programming, ask yourself, “Am I hiding bugs in this code by using defensive programming?” If you might be, add assertions to alert you to those bugs.

Don’t hide bugs when you program defensively.

TWO ALGORITHMS ARE BETTER THAN ONE

Checking for bad inputs and flawed assumptions is only part of what you can do to trap bugs in your programs. Just as another function can pass garbage to your function, yours can return garbage to its caller. You never want to do that.

Since both memcpy and memset simply return one of their parameters, there’s little chance that you’d accidentally return garbage in those routines. But in a more complex routine, you might not be so sure of your results.

For example, years ago, I had to write a 68000 disassembler as part of a development tool for Macintosh programmers. Speed wasn’t critical for the disassembler, but it was vital that it work correctly, so I chose to implement the code using a simple table-driven algorithm that I could easily test. I also used assertions to automatically catch any bugs that I missed while testing the code.

If you’ve ever looked at an assembly language reference book, chances are good that it described every instruction in painstaking detail. And as a part of that thoroughness, it showed a bit-pattern for each instruction. For instance, if you looked up the ADD instruction in a 68000 reference manual, you would see that it has this bit-pattern:

ADD:

You can ignore the register and mode fields of this instruction—we’re interested in only the bits that are explicitly 0 or 1—in this case, the upper 4 bits of the instruction. You figure out whether you have an ADD instruction by stripping away the nonexplicit bits and checking to see whether the upper 4 bits are 1101, or hex 0xD:

if ((inst & ØxFØØØ) == ØxDØØØ)

it’s an ADD instruction...

The DIVS instruction (used for signed division) has 7 explicit bits in its pattern:

DIVS:

Again, if you just strip out the nonexplicit register and mode fields, you can tell whether you have a DIVS instruction by using

if ((inst & ØxF1CØ) == Øx81CØ)

it’s a DIVS instruction...

You can use this mask-then-test technique to isolate every assembly language instruction, and once you know you’ve got an ADD, or a DIVS, you can call a decode function to make sense of those register and mode fields that we’ve been ignoring.

That’s how the disassembler worked in the tool I developed.

Of course, I didn’t have 142 different if statements to check for every possible instruction. Instead, I had a table containing a mask, pattern, and decode function for each instruction. I then wrote code to loop over this table, and if it matched an instruction, it called the corresponding routine to decode the register and mode fields.

Here’s a part of that table, and the code that uses it:

/*  idInst is a table of masks and patterns that

*  identify a bit-pattern as a specific type of

*  instruction.

*/

static identity idInst[] =

{

/* mask,   pat,    function     */

{ ØxFFØØ, ØxØ6ØØ, pcDecodeADDI  },

{ ØxF13Ø, ØxD1ØØ, pcDecodeADDX  },

{ ØxFØØØ, ØxDØØØ, pcDecodeADD   },

{ ØxFØØØ, Øx6ØØØ, pcDecodeBcc   },

{ ØxF1CØ, Øx418Ø, pcDecodeCHK   },

{ ØxF138, ØxB1Ø8, pcDecodeCMPM  },

{ ØxFFØØ, ØxØCØØ, pcDecodeCMPI  },

{ ØxF1CØ, Øx81CØ, pcDecodeDIVS  },

{ ØxF1ØØ, ØxB1ØØ, pcDecodeEOR   },

⋮

{ ØxFFØØ, Øx4AØØ, pcDecodeTST   },

{ ØxFFF8, Øx4E58, pcDecodeUNLK  },

{ ØxØØØØ, ØxØØØØ, pcDecodeError }

};

/*  pcDisasm

*

*  Disassemble one instruction and fill in the opc opcode

*  structure. pcDisasm returns an updated program counter

*  to the next instruction in the code.

*

*  Typical use: pcNext = pcDisasm(pc, &opc);

*/

instruction *pcDisasm(instruction *pc, opcode *popcRet)

{

identity    *pid;

instruction inst = *pc;

for (pid = &idInst[Ø]; pid->mask != Ø; pid++)

{

if ((inst & pid->mask) == pid->pat)

break;

}

return(pid->pcDecode(inst, pc+1, popcRet));

}

As you can see, pcDisasm is not a large function. It uses a simple algorithm that reads the current instruction, picks it out of the table, and then calls a decode routine to fill in the opcode structure that popcRet points to. As a final task, pcDisasm returns an updated program counter. This is necessary because not all 68000 instructions are the same length. The decode routines would, if necessary, read the extra parts of an instruction and then return the new program counter to pcDisasm, which passes it on.

Now back to the original point that you can’t always be sure your routines aren’t returning garbage.

With a function such as pcDisasm, it’s hard to tell whether you’re returning valid data. Even though pcDisasm itself might be properly identifying an instruction, the decode routines could be spewing garbage and you’d have a tough time spotting that. One way to trap such bugs is to put assertions in every decode routine. I’m not saying you shouldn’t do that, but an even more powerful approach would be to put the assertions into pcDisasm since it is the bottleneck routine for all of the decode functions.

The question is, How?

How would you automatically check that the decode routines are correctly filling in the opcode structure? You’d have to write code to validate that structure. And how would you do that? Well, basically, you’d have to write a routine that compared a 68000 instruction to the contents of an opcode structure. In other words, you’d have to write a second disassembler.

Now I know that may sound crazy, but is it really?

Because speed is critical to the success of a spreadsheet, Microsoft Excel uses a complex algorithm to make sure that it never recomputes a formula in a cell in which it doesn’t need to. The only problem was that because the algorithm was so complex, it was hard to modify without introducing bugs.

Not good…

To help catch bugs in the smart, fast, recalculation engine, the Excel team wrote a second, simple and brute-force recalculation engine that ran only in the debug of the program. After the smart engine finished its calculations, the second debug version of the recalculation engine would kick in and slowly but thoroughly recompute every cell that had a formula in it. An assertion would fire if there was any difference between the results of the two recalculation engines.

Microsoft Word had a similar problem. Because speed is also critical in the page layout code of a word processor, the Word programmers wrote the layout code in hand-tuned assembly language. That was great for speed, but it was lousy in terms of keeping bugs out of the code. And, unlike the code for Excel’s recalculation engine, which didn’t change very often, the layout code for Word changed regularly as new features were added. To automatically catch layout bugs, the Word programmers wrote C versions of every hand-tuned assembly language routine. If the two layout engines produced a different result, an assertion would fire.

Today’s web browsers could employ something similar: write one display engine that is highly optimized using every speed trick in the book, verified by a simple, brute-force debug display engine that produced its own output in an internal buffer. If the outputs of the two display engines didn’t match, a well-placed assertion could alert the development team to the problem.

In the same way, it made sense to use a debug-only disassembler to validate the primary disassembler in the tool I was working on.

I won’t bore you with the details of how I implemented pcDisasmAlt, the second disassembler, but it was logic rather than table driven. In brief, I used nested switch statements to successively peel away significant bits until I had isolated the exact instruction. This next piece of code shows how I used pcDisasmAlt to validate the primary disassembler:

instruction *pcDisasm(instruction *pc, opcode *popcRet)

{

identity    *pid;

instruction inst = *pc;

instruction *pcRet;

for (pid = &idInst[Ø]; pid->mask != Ø; pid++)

{

if ((inst & pid->mask) == pid->pat)

break;

}

pcRet = pid->pcDecode(inst, pc+1, popcRet);

#ifdef DEBUG

{

opcode opc;

/*  Check both outputs for validity  */

ASSERT(pcRet == pcDisasmAlt(pc, &opc));

ASSERT(compare_opc(popcRet, &opc) == SAME);

}

#endif

return (pcRet);

}

Normally, you should slip your debug checks into existing code without their getting in the way. I couldn’t quite manage that here—I had to create the pcRet local variable so that I could validate the pointer that pid‑>pcDecode returns.

This is OK since it doesn’t violate the fundamental principle that “You should always execute debug code in addition to and not instead of ship code.”

That may seem blatantly obvious right now, but once you start using assertions and debug code you’ll find that there are times when you’d rather execute debug code instead of ship code. We’ll see an example of this in Chapter 3, but for now, let me say this: Resist the urge. I had to modify pcDisasm to make the debug checks, but all the ship code is still executed.

I won’t pretend that you should write two versions of every function you have in your program. I do believe that most programs have key functionality that must work no matter what else goes wrong. In a spreadsheet it’s the recalculation engine. In a word processor or web browser, it’s the page layout engine. In a project manager, it’s the task scheduler. In a database, it is the search/extraction engine. And in every program, it’s the code that guarantees that the user will never lose data.

As you write code, keep an eye out for opportunities to validate your results. Bottleneck routines are particularly good places in which to look. Also, be sure to use a different algorithm, if possible, and not just a second implementation of the same algorithm. By using a different algorithm, you not only find implementation bugs, but you also increase your odds of finding bugs in the algorithms themselves.

Use a second algorithm to validate your results.

STOP BUGS AT THE STARTING LINE

Up to now, I’ve been ignoring the register and mode bits of instructions, but what if special encodings for those fields change the underlying instruction? For example, the pattern for the exclusive-or EOR instruction looks like this:

EOR:

and the pattern for the CMPM instruction looks remarkably similar:

CMPM:

Notice that if the Effective Address Mode field of the EOR instruction is 001, the EOR instruction will look like the CMPM instruction. The problem, of course, is that if EOR is placed earlier in the idInst table, it will incorrectly pluck out any CMPM instructions that come through.

The good news is that because pcDisasm and pcDisasmAlt use different algorithms, you should get an assertion failure the first time you disassemble a CMPM instruction. This happens because pcDisasm will incorrectly fill the opcode structure with an EOR instruction, but pcDisasmAlt will correctly (we hope) fill it with a CMPM instruction. When the two structures are compared in the debug code, you will get an assertion failure. This is an example of the power of using different algorithms in your debug functions.

The bad news is that you’ll catch this bug only if you try to disassemble a CMPM instruction. I’d like to think that your external test suites would be thorough enough to catch this bug, but remember what I said in Chapter 1: You want to catch bugs automatically, at the earliest possible moment, and without relying on the skill of others.

So while you could push this off onto your testing group, don’t. Despite what many programmers believe, testers are not there to test your code. That’s your responsibility. If you disagree, just name one other job in which it’s acceptable to do slipshod work simply because somebody else is going to review it for mistakes. Why should programming be an exception? If you want to consistently write bug-free code, you must grab the reins and take charge. So let’s start here.

Whenever you notice something risky about your code, ask yourself, “How can I automatically catch this bug at the earliest possible moment?” By habitually asking yourself this question, you’ll find all sorts of ways to make your programs more robust.

You can detect bugs in the table by scanning it during startup initialization in the debug version of your program. You do this by looking at every table entry to verify that no earlier entry will incorrectly intercept its instructions. The code to check the table for such bugs is short, but it’s not necessarily clear:

void CheckIdInst(void)

{

identity *pid, *pidEarlier;

instruction inst;

/*  For each instruction in the table... */

for (pid = &idInst[Ø]; pid->mask != Ø; pid++)

{

/*  ...verify that no earlier entries collide with it. */

for (pidEarlier = &idInst[Ø]; pidEarlier < pid; pidEarlier++)

{

inst = pid->pat | (pidEarlier->pat & ~pid->mask);

if ((inst & pidEarlier->mask) == pidEarlier->pat)

ASSERT(bitcount(pid->mask) < bitcount(pidEarlier->mask));

}

}

}

This check works by comparing the current instruction with each of the instructions that appear earlier in the table. All of the instructions have “don’t care” bits—they are the register and mode bits that are masked out. But what if those “don’t care” bits just happen to form the bit-pattern for an earlier instruction in the table? In that case, you would have a collision between two table entries. Which entry should be placed earlier in the table?

The answer is simple.

If two entries in the table match the same instruction, the entry with more explicit bits should appear earlier in the table. If this isn’t intuitively evident to you, take another look at the patterns for EOR and CMPM. If both of those patterns matched the same instruction, which would you choose as the “right” match? Why? Since the masks have one bit set for each explicit 0 or 1 bit in the pattern, you can tell which entry is more specific by comparing the number of set bits in each mask.

To determine whether two instructions collide, you take the pattern of one entry and force its “don’t care” bits to exactly match the pattern of each of the earlier entries. This is the value that is assigned to inst in the code. By design, inst has to match the pattern of the current entry since you’re changing only bits that don’t matter, but if inst also matches the pattern of an earlier entry, the two collide and you must compare the masks. The entry earlier in the table must have more bits in its mask, or the wrong instruction will be chosen from the table.

By calling CheckIdInst during program startup, you will catch collision bugs the very first time you execute the program—you won’t need to disassemble a single instruction. You should look for similar types of startup checks in your own code because they can quickly alert you to bugs that otherwise might go unnoticed for some time.

Don’t wait for bugs to happen; use startup checks.

ASSERTIONS ARE FOREVER

In this chapter, you’ve seen how you can use assertions to automatically catch bugs in your code. And while this is a valuable tool that can help you find that “last” bug much sooner than before, you can overuse it as you can any tool. It’s up to you to determine where to draw the line.

I know that if I were an astronaut about to board a rocket into space, I would be silently praying that the programmers who wrote the rocket’s software had taken every opportunity to use assertions and defensive programming.

But when I’m playing Angry Birds, I simply don’t care if the game crashes—not that it ever has. Angry Birds just doesn’t need the same level of defense as the space program.

Use your best judgment on how much is “too much” or “too little.”

Another thing: In case it’s not obvious, you should keep assertions in your code for the life of the project—don’t remove them from your source code once you’ve shipped your program. Those assertions will continue to be valuable for all future versions of your masterpieces.

As of this second edition of Writing Solid Code, a curious shift in philosophy among programmers is taking hold—programmers are choosing to release their code to end users with all assertions live and active. They’re moving away from the idea of building both a DEBUG and ship version of their projects. DEBUG and ship versions are one and the same.

Twenty years ago when the first edition of Writing Solid Code was released, processors were so slow and memory so scarce, I couldn’t imagine shipping code with the assertions still intact. That would have absolutely killed your product in the marketplace. But today, because machines have enormous amounts of memory and processors that run at blistering speeds, programmers are choosing to ship code with their assertions intact and use crowdsourcing to help find every possible bug in their code.

I’m sure you’ve seen it in action: You’re happily using a program and suddenly a message pops up saying, “An internal error has occurred. Press the Report button to help us correct this in future versions; otherwise press Cancel to continue.”

That’s an assertion firing somewhere in the program, albeit with a much friendlier alert mechanism than simply aborting with an error message. Why not take advantage of it?

QUICK REVIEW

◆

Assertions are a shorthand way to write debugging checks. Use them to catch illegal conditions that should never arise in bug-free code. And by “never,” I don’t mean “rarely.” Never means never. Don’t confuse illegal conditions with error conditions, which you must handle in the shipped product, however rarely they occur.

◆

Use assertions to validate function arguments and to alert programmers when they attempt to use behavior that is undefined. The more rigidly you define your functions, the easier it is for you to validate the arguments.

◆

Once you’ve written a function, review it and ask yourself, “What am I assuming?” If you find an assumption, either assert that your assumption is always valid, or rewrite the code to remove the assumption. Also ask, “What is most likely to be wrong in this code, and how can I automatically detect the problem?” Strive to implement tests that catch bugs at the earliest possible moment.

◆

Textbooks encourage programmers to program defensively, but remember that this coding style hides bugs. When you write defensive code, use assertions to alert you if the “can’t happen” cases do happen. You get the best of both worlds—ship code that self-corrects, and debug code that alerts you to a bug.

THINGS TO THINK ABOUT

1.

Suppose you distribute debug versions of a shared library and you want to include assertions, but the people using the library don’t have access to the library’s source code. If an assertion failed in the library, displaying the file name and line number in the library’s sources would be meaningless. How could you define an assertion macro, ASSERTMSG, that would display a meaningful error message instead of the file name and line number? For instance, the memcpy routine might display this assertion:

Assertion failure in memcpy: The blocks overlap

2.

When programmers add new elements to an enumeration, they sometimes forget to add new cases to the appropriate switch statements. How could you use assertions to detect this problem?

3.

CheckIdInst verifies that the idInst table entries are in the correct order, but out-of-order entries aren’t the only kind of problem that can occur in the table. With so many numbers in the table, it would be easy to mistype the value of a mask or pattern. How could you enhance CheckIdInst to help automatically catch typing mistakes?

4.

Earlier we saw that when the Effective Address Mode field of the EOR instruction is 001, it is really a CMPM instruction. There are other restrictions in the EOR instruction. For example, the 2-bit mode field can never be 11 (that would make it a CMPA.L instruction), and if the Effective Address Mode field is 111, the Effective Address Register field must be either 000 or 001. Because the pcDecodeEOR function should never be called with these non- EOR combinations, how would you add assertions to it to catch bugs in the table?

5.

How could you use a second algorithm to verify the qsort function? How could you verify a binary search routine? And how about the itoa function?

PROJECT: Contact the company who wrote your operating system and encourage them to provide a debugging version for programmers. This, by the way, is good for both parties because OS companies want people to write applications for their operating systems. It’s in their best interest to make it easier to bring bug-free products to market.

3

FORTIFY YOUR

SUBSYSTEMS

You can have 50,000 fans attending a sporting game, but you need only a handful of people to check tickets—provided, of course, that they’re positioned at the entry gates. Your program has such gates; they’re the entry points to your subsystems.

Think of the file system you use. You open files, close files, read and write files, and create files. That’s five basic operations, but the code to support those operations is often large and complex. You use the entry points without worrying much about file directories, free space maps, or how to read from or write to a specific hardware device whether it’s a disk drive, an optical drive, a flash drive, a network connection, or some other device.

What about a memory manager? You allocate memory, release it, and sometimes change its size. But again, there can be a lot of support code behind those operations. If you’re using a language such as C# or Java that handles all of your allocations and garbage collection for you, you have even fewer points of entry, and there’s a lot more code behind the scenes supporting all that functionality.

In general, a subsystem hides its implementation details, which can be quite complex, and instead provides a few key entry points that programmers use to communicate with the subsystem. If you were to add some debugging checks to the entry points of such a subsystem, you could get substantial error checking without having to make many, if any, changes to the rest of the subsystem’s implementation.

Anyone reading this book should be familiar with C’s malloc, free, and realloc memory subsystem to handle memory allocation. You could fill the subsystem with assertions, you could thoroughly test it, and you could write a superb programmer’s guide. But you and I both know that programmers will still have problems when they use C’s memory subsystem.

Here’s a suggestion: When you’ve finished writing a subsystem, ask yourself, “How are programmers going to misuse this subsystem, and how can I detect these problems automatically?” Ideally you would have asked this question before you began coding, to eliminate risky designs, but ask it again anyway. For a memory manager, you can rely on programmers to:

◆

Allocate a block and use the uninitialized contents.

◆

Free a block, but continue to reference the contents.

◆

Call realloc to change the size of a block and when it moves to continue to reference the contents at the old location.

◆

Allocate a block but “lose” it because its pointer isn’t saved.

◆

Read or write beyond the boundaries of a block.

◆

Fail to notice error conditions.

These are not wildly hypothetical problems—they show up all the time if you’re using C or C-derived languages. Worse, they’re usually hard to spot because they aren’t repeatable. You crash once, never to see the problem again—at least not until one of your users calls you in a huff and asks you to please fix the bug that keeps clobbering him or her in some common scenario.

Memory bugs are hard to spot, but that doesn’t mean you can’t improve things. Assertions are worthwhile, but you have to execute them if they’re to report problems. Look at the problems above and tell me how assertions in your memory manager would catch them. They can’t.

In this chapter, I’m going to talk about techniques you can use to ferret out subsystem bugs that would otherwise be difficult to find. I’ll be beefing up C’s memory manager in this chapter because the C memory manager is so well known and so well understood.

And although I’ll be using the C memory manager as the example in this chapter, if you’re writing code in C#, Java, or any other language that handles memory management automatically for you, don’t mentally check out here because C’s memory manager doesn’t apply to you for day-to-day programming. This chapter is not about the C memory manager. It’s about how to take any subsystem, in any program, and turn it into a powerful tool for detecting bugs automatically in your code.

For C purists reading this, the code in this chapter is not portable at times because C’s pointer model is not portable across all architectures. For simplicity, I’ll be assuming a ”flat” pointer architecture in the examples to convey the underlying philosophies.

SUBSYSTEM CHECKS

Normally, you would build subsystem checks directly into your subsystems, but I’m not going to do that here for two reasons.

The first reason is that I don’t want to clutter the book’s examples with the code for implementing malloc, free, and realloc. The second reason is that you often don’t have the source code for the subsystems that you use.

Instead of building the tests into sources that you may not have, or that are undoubtedly different from what I have, I’m going to put scaffolding around the entry points to the subsystem. This is, after all, what you would have to do if you don’t have the source code to a subsystem that you’re using.

And while I’m at it, I’m going to adopt the naming convention used throughout the rest of this book.

Let’s start with the wrapper function for malloc. It looks like this:

/*  fNewMemory -- allocate a memory block  */

flag fNewMemory(void **ppv, size_t size)

{

byte **ppb = (byte **)ppv;

*ppb = (byte *)malloc(size);

return (*ppb != NULL);             /*  Success? */

}

This may look more complicated than malloc, but that’s largely because of the noise introduced by the void ** argument pointer. If you look at how a programmer would call this function, you can see that it is as clear, if not more clear, than a call to malloc. Instead of writing:

if ((pbBlock = (byte *)malloc(32)) != NULL)

successful -- pbBlock points to the block

else

unsuccessful -- pbBlock is NULL

you would write

if (fNewMemory(&pbBlock, 32))

successful -- pbBlock points to the block

else

unsuccessful -- pbBlock is NULL

which would achieve the same thing. The only difference between the two functions is that fNewMemory separates the “success” and “pointer” outputs whereas malloc fuses them into one dual-purpose output. In both cases,pbBlock points to the block if it was allocated or is NULL if it wasn’t.

In the last chapter I said that you should either eliminate undefined behavior or use an assertion to verify that it doesn’t happen. If you apply that advice to malloc, you should see that there are two undefined items you have to handle. First, it’s undefined behavior, according to the ANSI C standard, if you call malloc to allocate a zero-length block. Second, if malloc returns a block, it leaves the contents uninitialized—the block may be filled with zeros, or it may contain random garbage. You just don’t know.

Handling a request to allocate a zero-length block is simple; you check for it with an assertion to automatically alert you to undefined behavior

But what about the other problem? Uninitialized memory…

How can you assert that the block contents are, or are not, valid? It doesn’t make sense. That leaves you with just one option: Eliminate the undefined behavior. The obvious approach is to have fNewMemory zero-fill blocks the moment it allocates them. That would work, but in a correct program, the contents of a newly allocated block shouldn’t matter. Burdening your ship program with unnecessary fills is something you should avoid.

Unnecessary fills can also hide bugs.

Suppose you allocate memory for a data structure but you forget to initialize one of the fields—or more realistically, a maintenance programmer adds a new field to an existing data structure but forgets to add the initialization code for that new field. That’s a bug, but you may not notice it if fNewMemory sets the field to 0 or any other possibly useful value.

Still, you don’t want to leave the contents undefined because that makes bugs hard to reproduce. What if a bug showed up only when the garbage happened to be one particular value? You would miss the bug much of the time and periodically crash for no apparent reason. Imagine how hard it would be to reach zero-bugs if every bug happened only some of the time—programmers (and testers) would go crazy trying to track down problems.

The key to exposing bugs is to eliminate random behavior wherever you find it.

Exactly how you do this depends on the subsystem and the random behavior involved. For malloc you can eliminate randomness by filling the block—but only in debug versions of your program. That solves the problem without putting a ball and chain on the code you ship. But remember, you don’t want to hide bugs. The idea is to fill the blocks using a weird value that looks like garbage but that makes bugs apparent.

When I was working on Macintosh Excel in the mid-80s, I used the value 0xA3 to fill newly allocated memory. I chose this value by asking a number of questions: What would force a bad pointer to show itself? What about a bad counter or index? What if the block were executed?

The Macintosh models at the time used Motorola’s 68000 processor which would crash if you tried to reference a 16-bit or 32-bit value using an odd-valued pointer, so I knew the fill value had to be odd. I also knew that I was more likely to spot a bad counter or index if it were large and caused noticeable delays or forced the system to misbehave. Finally, of all the weird-looking, odd-valued, and large numbers you can represent in a byte, I chose 0xA3 because if the block were somehow executed, the undefined 68000 machine language instruction 0xA3A3 would instantly crash in a clean, predictable way—you’d get an “undefined A-Line trap” error in the system debugger. This last point may seem nit-picky, but why not use every opportunity, however remote, to automatically catch bugs?

The value you choose for your target processor may be different, but the process for choosing the value is similar: You ask yourself how uninitialized data could be used and then look for ways to make it noticeable.

If you add both the size-check assertion and the code to fill undefined memory, fNewMemory becomes

#define bGarbage ØxA3

flag fNewMemory(void **ppv, size_t size)

{

byte **ppb = (byte **)ppv;

ASSERT(ppv != NULL  &&  size != Ø);

*ppb = (byte *)malloc(size);

#ifdef DEBUG

{

if (*ppb != NULL)

memset(*ppb, bGarbage, size);

}

#endif

return (*ppb != NULL);

}

Not only does this version of fNewMemory help make bugs reproducible, but it often makes them easier to track down. If you find yourself staring at a loop index with the value 0xA3A3, or a pointer with the value 0xA3A3A3A3, it’s clear that you’re looking at uninitialized data. More than once, I’ve been tracking down one bug, only to spot another in the process because I ran across some unexpected combination of 0xA3s.

So look at the subsystems in your application and isolate the design points that can cause random bugs. Once you have identified these points, either remove them by changing your design, or add debugging code to minimize the amount of random behavior.

Eliminate random behavior.

Force bugs to be reproducible.

SHRED YOUR GARBAGE

The wrapper function for free looks like this:

void FreeMemory(void *pv)

{

free(pv);

}

The ANSI C standard says that what free does is undefined if you pass it an invalid pointer. That sounds reasonable, but how can you tell whether pv is valid? How can you assert that pv points to the start of an allocated block? The answer is that you can’t, at least not without more information.

It gets worse.

Suppose your program maintains a tree of some sort and that the deletenode routine calls FreeMemory to release one of the nodes. What will happen if there is a bug in deletenode that causes it to release the node but fails to update the link pointers in the surrounding allocated nodes? Obviously, you will have a tree structure that contains a free node. But guess what? On many systems, that free node will still look valid.

That shouldn’t be too surprising. When you call free, you’re telling the memory manager that you don’t need the memory anymore, so why should it waste time scrambling the contents?

That’s a reasonable optimization, but it has the nasty side effect that free memory—which is garbage—can look like it contains valid data. Instead of having a tree with a node that will crash the code the next time you traverse the structure, you could have a tree that still looks valid even though it isn’t. How likely are you to spot that problem? Unless you have the luck of a lotto winner, not very.

“No problem,” you say, “I’ll just add some debug code to FreeMemory to fill the block with bGarbage (0xA3) before it calls free. That way the contents are guaranteed to look like garbage and the tree manipulation routines will break when they hit the free node.” Good idea, but how large is the block? Oops, you don’t know that either.

You could throw up your hands at this point and declare that FreeMemory has beaten you. After all, you can’t assert that pv is valid since you have no way to do it, and you can’t destroy the contents of the block because you don’t know how large it is.

Instead of giving up, let’s assume for a moment that you have a debug function, sizeofBlock, that will give you the size of any allocated memory block. If you have the source code to your memory manager, you could probably write such a routine without much effort. But even if you don’t, don’t worry—I’ll provide an implementation for sizeofBlock later in the chapter.

Using sizeofBlock, you can destroy the memory before you release it:

void FreeMemory(void *pv)

{

ASSERT(pv != NULL);

#ifdef DEBUG

{

memset(pv, bGarbage, sizeofBlock(pv));

}

#endif

free(pv);

}

This code not only fills the block, but it also validates pv as a side effect of calling sizeofBlock. If the pointer is bad, sizeofBlock will assert—it can do this because, obviously, it must know about every allocated block.

It might seem strange that I used an assertion to verify that pv is not NULL even though NULL is a legal argument for C’s free function—the ANSI C standard says that free does nothing in this case.

The reason I included the assertion shouldn’t be too surprising: I don’t believe in passing arguments to functions where it has no meaning but those conjured up for convenience. Why would you ever try to free an unallocated block? That makes no sense. In my mind, that’s clearly bug in the calling routine. The assertion simply validates this philosophy.

realloc is another function that releases memory and creates garbage. Here is its “wrapper” function:

flag fResizeMemory(void **ppv, size_t sizeNew)

{

byte **ppb = (byte **)ppv;

byte *pbNew;

pbNew = (byte *)realloc(*ppb, sizeNew);

if (pbNew != NULL)

*ppb = pbNew;

return (pbNew != NULL);

}

Like fNewMemory, fResizeMemory returns a status flag to indicate whether it successfully changed the size of the block. Assuming that pbBlock points to allocated memory, you could resize the block this way:

if (fResizeMemory(&pbBlock, sizeNew)

successful -- pbBlock points to the new block

else

unsuccessful -- pbBlock points to the old block

You should note that, unlike realloc, fResizeMemory does not return a null pointer if the operation fails; it returns the original pointer, which still points to the allocated, though unchanged, block.

The realloc function (and so fResizeMemory) is interesting in that it contains elements of both free and malloc, depending on whether you’re shrinking a block or expanding it. In FreeMemory, I destroyed the block contents just before the block was released. In fNewMemory, I filled the new block with weird-looking “garbage” right after calling malloc. You must do both to make fResizeMemory robust. That requires two separate blocks of debug code:

flag fResizeMemory(void **ppv, size_t sizeNew)

{

byte **ppb = (byte **)ppv;

byte *pbNew;

#ifdef DEBUG

size_t sizeOld;

#endif

ASSERT(ppb != NULL  &&  sizeNew != Ø);

#ifdef DEBUG

{

sizeOld = sizeofBlock(*ppb);

/*  If shrinking, erase the tail contents. */

if (sizeNew < sizeOld)

memset((*ppb)+sizeNew, bGarbage, sizeOld-sizeNew);

}

#endif

pbNew = (byte *)realloc(*ppb, sizeNew);

if (pbNew != NULL)

{

#ifdef DEBUG

{

/*  If expanding, initialize the new tail. */

if (sizeNew > sizeOld)

memset(pbNew+sizeOld, bGarbage, sizeNew-sizeOld);

}

#endif

*ppb = pbNew;

}

return (pbNew != NULL);

}

That might look like a lot of extra code, but if you look closer you’ll see that most of the code is whitespace, braces, #ifdef directives, and comments.

But even if you felt that you needed to add a ton of extra code to make important DEBUG tests, worrying about the code to implement those tests would be unnecessary unless your DEBUG tests are so slow, and take so much time, that the programmers and testers on your team won’t use them.

Let’s not stop here in our effort to strengthen the DEBUG version of the C memory management subsystem.

When looking at any subsystem, you should look at every aspect of how that subsystem is used and try to isolate bugs based on that usage.

For example, suppose that instead of freeing a tree structure node, your program called fResizeMemory to expand the node to accommodate a variable-length data structure. If fResizeMemory moved the node as it expanded it, you would now have two nodes: the real one at the new location, and the untouched garbage left at the old location.

What would happen if the programmer who wrote expandnode wasn’t aware that fResizeMemory could move the node as it expanded it? Wouldn’t that programmer leave the tree structure in its old state, with surrounding nodes still pointing to the original unexpanded and valid-looking block? And wouldn’t the new block end up floating out in memory space with nothing pointing to it? In effect, you would have a valid-looking but flawed tree structure—and a lost block of memory. That’s not good.

You might think that fResizeMemory could destroy the original memory any time it moved a block while expanding it. A simple call to memset would do the trick:

flag fResizeMemory(void **ppv, size_t sizeNew)

{

⋮

pbNew = (byte *)realloc(*ppb, sizeNew);

if (pbNew != NULL)

{

#ifdef DEBUG

{

/*  If the block moved, destroy the old one. */

if (pbNew != *ppb)

memset(*ppb, bGarbage, sizeOld);

/*  If expanding, initialize the new tail. */

if (sizeNew > sizeOld)

memset(pbNew+sizeOld, bGarbage, sizeNew-sizeOld);

}

#endif

*ppb = pbNew;

}

return (pbNew != NULL);

}

Unfortunately, you can’t do that. Even though you know the size and location of the old block, you can’t scramble the contents because you don’t know what the memory manager does with its free memory. Some memory managers don’t do anything with the memory, but others use it to store free-chain information, or other internal implementation data. The fact is that once you release memory, you don’t own it, so you can’t touch it. If you do, you risk corrupting your system.

Maybe you’re thinking, “I’ve covered all the other cases, so is this one worth worrying about?” I think so, mainly because C programmers either don’t know, or regularly forget, that realloc can move blocks. Detecting this problem is important.

In one extreme case, when I was adding features to Microsoft’s internal 68000 cross assembler, I was asked by the Macintosh Word and Excel programmers to track down a long-time bug that would randomly crash the system during the build process. The only difficulty was that the bug showed up about as often as you get your hair cut. Individuals wouldn’t get hit too often, but the bug was constantly biting somebody, and that gave it some priority. I’ll spare you the details, but it took weeks, off and on, to come up with a reproducible scenario for the bug, and then it took three days to track down the actual cause.

That’s a long time to find the cause of a reproducible bug, but I had no idea what was causing it, and every time I stepped through the data structures, they looked perfect. I had no idea that those perfect data structures were, in fact, garbage left by an earlier call to realloc that moved the block to a different memory location but left the original data intact.

The real problem was not that it took me so long to find the exact cause of the bug, but rather that it took so much effort to come up with a reproducible case. Not only did realloc have to move the block of memory as it was expanding it, but the old memory had to be reallocated and filled with new data in order for the bug to occur. In the assembler, both happened rarely.

This brings up another guideline for writing bug-free code: You don’t want anything to happen rarely, at least not in the DEBUG version of your program. You need to isolate those behaviors in your subsystems that may happen and make sure that they do happen. And often. If you find rare behavior in your subsystems, be sure to do something—anything—to stir things up.

The assembler bug could have been found within hours instead of years if realloc hadn’t so rarely moved blocks when it expanded them.

So, the question is, how can you force C’s realloc function to move blocks more frequently?

And, of course, the answer is that you can’t.

But you can simulate what realloc does.

If a programmer calls fResizeMemory to expand a block, the DEBUG version of fResizeMemory can always move the existing block by first creating a new block, copying the contents of the old block to the new block, and finally releasing the old block. The DEBUG code would do exactly what realloc does:

flag fResizeMemory(void **ppv, size_t sizeNew)

{

byte **ppb = (byte **)ppv;

byte *pbNew;

#ifdef DEBUG

size_t sizeOld;

#endif

ASSERT(ppb != NULL  &&  sizeNew != Ø);

#ifdef DEBUG

{

sizeOld = sizeofBlock(*ppb);

/*  If the block is shrinking, pre-fill the soon-to-

*  be-released memory. If the block is expanding,

*  force it to move (instead of expanding in place)

*  by faking a realloc.

*/

if (sizeNew < sizeOld)

memset((*ppb)+sizeNew, bGarbage, sizeOld-sizeNew);

else if (sizeNew > sizeOld)

{

byte *pbForceNew;

if (fNewMemory(&pbForceNew, sizeNew))

{

memcpy(pbForceNew, *ppb, sizeOld);

FreeMemory(*ppb);

*ppb = pbForceNew;

}

}

}

#endif

pbNew = (byte *)realloc(*ppb, sizeNew);

⋮

}

Here I’ve added new code that is executed only if the block is expanding. By allocating the new block before releasing the old block, you know that the block will be moved unless, of course, the allocation fails. If that happens the new code behaves like a big no-op instruction.

But notice what I’ve done here. Not only does the code force the block to move regularly, but—as a side effect—the code destroys the contents of the old block. That happens when it calls FreeMemory to release the original block.

Now maybe you’re wondering, “Since the code fakes a realloc, why does it still call realloc?” After all, you could speed things up by embedding a return statement in the new code to fResizeMemory:

if (fNewMemory(&pbForceNew, sizeNew))

{

memcpy(pbForceNew, *ppb, sizeOld);

FreeMemory(*ppb);

*ppb = pbForceNew;

return (TRUE);               /*  Return  */

}

You could do that, but don’t—it’s a bad practice to get into. Remember that debug code is extra code, not different code. Unless there is a compelling reason not to, you should always execute the ship code, even if it’s redundant. After all, there is no better way to catch bugs in code than to execute it, and you want to execute ship code as much as possible.

Sometimes when I explain the concepts in this section to a programmer, he or she will argue that always moving memory is just as bad as never moving it—that I’ve gone to the other extreme. That’s an astute observation and is worth talking about for a moment.

Always doing something would be as bad as never doing it if it were true for both the ship version and the debug version of your program. In this example, though, the ship version of fResizeMemory is practically catatonic while the debug version moves blocks with such zeal that you’d think it was on amphetamines.

It’s OK if something happens rarely, as long as it’s not rare in both the ship version and debug version of your program.

If something happens rarely in the ship version, force it to happen often in the debug version.

KEEP A JOURNAL TO JOG YOUR MEMORY

The problem with C’s memory manager—from a debugging viewpoint—is that you know the size of a block when you first create it but you lose that information almost immediately unless you keep a record of it somewhere. You’ve already seen how valuable the sizeofBlock function can be, but imagine how useful it would be to know how many blocks are allocated and where they fall in memory. If you knew that, I could hand you an arbitrary pointer and you could tell me whether it was valid. Consider how useful that would be, particularly for validating pointer arguments to functions.

Suppose you had a function fValidPointer that took a pointer and a size and returned TRUE if the pointer actually pointed to size bytes of allocated memory. You could then write special-purpose, more stringent versions of popular routines. For instance, if you found that you often fill parts of allocated memory, you could bypass the lenient memset function and instead call your own FillMemory routine, which would rigorously validate the pointer argument:

void FillMemory(void *pv, byte b, size_t size)

{

ASSERT(fValidPointer(pv, size));

memset(pv, b, size);

}

By calling fValidPointer, you ensure that pv points to a valid block and that there are at least size bytes from pv to the end of the block. That’s a far stronger test than memset’s null pointer check. This is an example of trading both size and speed for extra security.

What I’m saying is that if you keep extra information in the debug versions of your programs, you can often provide much stronger error checking. So far, I’ve shown you how to use sizeofBlock to fill memory in FreeMemory and fResizeMemory, but filling memory is a “weak” way to find bugs compared to what you can do if you keep a record of every allocated memory block.

Again, I’m going to assume the worst-case scenario: that you can’t derive any information about allocated memory blocks from the subsystem itself. For the memory manager, this worst case means that you can’t derive the size of a block, that you can’t tell whether a pointer is valid, and that you can’t even tell whether a block exists or how many there are. If you need this information, you have to provide it yourself, and that means keeping an allocation log of some sort. How you keep the log doesn’t matter, but you must have the information handy when it’s called for.

Here’s one possible way to maintain such a log: When you allocate a block in fNewMemory, allocate an extra block for a log entry; when you release a block in FreeMemory, release the log information; and when you change the size of block in fResizeMemory, update the log information to reflect the new size and location of the block. These three actions can be isolated in, not surprisingly, three debug interfaces:

/*  Create a memory record for the new block.  */

flag fCreateBlockInfo(byte *pbNew, size_t sizeNew);

/*  Release the information stored about a block.  */

void FreeBlockInfo(byte *pb);

/*  Update the information of an existing block.  */

void UpdateBlockInfo(byte *pbOld, byte *pbNew, size_t sizeNew);

How these routines maintain the log information isn’t too important, provided, of course, that they don’t slow the system down to the point that it’s unusable. You can find code in Appendix B that implements these memory log functions, along with implementations for sizeofBlock and fValidPointer.

Modifying FreeMemory and fResizeMemory to call the appropriate memory log routines is straightforward. FreeMemory becomes

void FreeMemory(void *pv)

{

#ifdef DEBUG

{

memset(pv, bGarbage, sizeofBlock(pv));

FreeBlockInfo(pv);

}

#endif

free(pv);

}

In fResizeMemory, you call UpdateBlockInfo if realloc successfully changes the size of the block. If realloc fails, there isn’t anything to update. The tail part of fResizeMemory becomes:

flag fResizeMemory(void **ppv, size_t sizeNew)

{

⋮

pbNew = (byte *)realloc(*ppb, sizeNew);

if (pbNew != NULL)

{

#ifdef DEBUG

{

UpdateBlockInfo(*ppb, pbNew, sizeNew);

/*  If expanding, initialize the new tail.  */

if (sizeNew > sizeOld)

memset(pbNew+sizeOld, bGarbage, sizeNew-sizeOld);

}

#endif

*ppb = pbNew;

}

return (pbNew != NULL);

}

Modifying fNewMemory is a bit more complicated, and that’s why I’ve saved it for last. When you call fNewMemory to allocate a block in the DEBUG version, fNewMemory must allocate two blocks: one for your request, and one for the log information. For the call to succeed both allocations must succeed; otherwise you’ll have a memory block with no log information to back it up. That’s important because without that log information you’ll an get assertion failure if you call any function that validates its pointer arguments.

In the following code, you’ll see that if fNewMemory allocates the memory requested but fails to allocate the memory for the log entry, it will release the first memory block and fake a memory failure. This keeps the memory system and the log information synchronized.

flag fNewMemory(void **ppv, size_t size)

{

byte **ppb = (byte **)ppv;

ASSERT(ppv != NULL  &&  size != Ø);

*ppb = (byte *)malloc(size);

#ifdef DEBUG

{

if (*ppb != NULL)

{

memset(*ppb, bGarbage, size);

/*  If unable to create the block information,

*  fake a total memory failure.

*/

if (!fCreateBlockInfo(*ppb, size))

{

free(*ppb);

*ppb = NULL;

}

}

}

#endif

return (*ppb != NULL);

}

That does it.

Now that you have a total record of the memory system, you can easily write functions like sizeofBlock and fValidPointer (see Appendix B), or anything else you find useful.

Keep subsystem debug information to allow for stronger assertion tests.

DON’T WAIT FOR BUGS TO CALL

Up to this point, every change I’ve suggested helps you notice bugs when they happen. That’s good, but it’s not automatic. Think about the deletenode routine I talked about earlier. If that code called FreeMemory to release a node and left dangling pointers in the tree, would you have any chance of spotting the problem if those pointers never get used? No, you wouldn’t.

Or, what if I’d introduced a bug in fResizeMemory because I forgot to call FreeMemory:

⋮

if (fNewMemory(&pbForceNew, sizeNew))

{

memcpy(pbForceNew, *ppb, sizeOld);

/*  FreeMemory(*ppb); */    /* Forgot this line */

*ppb = pbForceNew;

}

That would be a subtle bug because nothing obvious would go wrong. But every time you executed the code you would “lose” a memory block because the only pointer to it is destroyed when you assign pbForceNew to *ppb.

Will the DEBUG code help catch this bug? Not at all.

Bugs such as this one differ from the bugs I talked about earlier in that nothing illegal ever happens. Just as roadblocks are worthless if the crooks never try to leave town, the debug code I’ve shown so far is worthless for catching bugs when the data isn’t used. That doesn’t mean the bugs don’t exist. They do. It just means you can’t see them—they’re “lying low.”

To find these types of bugs, you do the programmer’s equivalent of a house-to-house search. Instead of waiting for the bugs to show themselves, you add debug code to actively search for these types of problems.

In deletenode’s case, you have a dangling pointer to a block that is no longer allocated. In fResizeMemory’s case, you have a block that is allocated but that has no pointer to it. These bugs would normally be hard to find, but not if you’ve been keeping debug information.

Think about how you find errors in your bank statement: You have a list of the funds you think you’ve allocated; the bank has a list it thinks you’ve allocated; you find errors by comparing the two lists. The way you find dangling pointers and lost blocks is no different. You compare the list of known pointers, which is stored in your data structures, to the list of known allocations, which is stored in the debug information. If you find pointers that don’t reference allocated blocks, or blocks that don’t have any pointers to them, you’ve got problems.

But programmers—especially experienced programmers—balk at the idea of checking every pointer stored in every data structure because tracking them down seems difficult, if not impossible. The reality, though, is that even poorly written programs clump pointers into classes of allocations.

For example, the compiler or interpreter for the language you commonly use might allocate memory for 753 symbol names, but it doesn’t keep track of them by means of 753 global variables. That would be silly. Instead it would use an array, or a hash table, or a tree, or possibly a simple linked list to store those names. There may be 753 symbol names, but looping over any of these data structures is simple and takes little code.

To compare the list of pointers stored in the data structures to the list of allocations stored in the debug information, I’ve defined three functions that work together with the information gathering routines described in the last section—you can again find their implementations in Appendix B:

/*  Mark all blocks as "unreferenced."  */

void ClearMemoryRefs(void);

/*  Note that the block pointed to by "pv" has a reference.  */

void NoteMemoryRef(void *pv);

/*  Scan the reference flags looking for lost blocks  */

void CheckMemoryRefs(void);

The way you use these routines is straightforward. First you call ClearMemoryRefs to set the debug information to a known state. Next, you scan your global data structures and call NoteMemoryRef for each pointer to allocated memory, both to validate the pointer and to mark that the block was referenced. Once you’ve accomplished that, every pointer should be validated and every block should have a reference mark. Finally, you call CheckMemoryRefs to verify that all blocks are marked; if CheckMemoryRefs finds an unmarked block, it will assert, alerting you to the lost block.

Let’s see how you would use these routines to validate the pointers in a binary tree that stores symbol table information in which each node in the tree looks like this:

/*  "symbol" is the node definition for a symbol name.

*  We allocate one of these nodes for every symbol

*  defined in the user’s assembly source code.

*/

typedef struct SYMBOL

{

struct SYMBOL *psymRight;

struct SYMBOL *psymLeft;

char *strName;

⋮

} symbol;                  /*  Naming: sym, *psym  */

I’ve shown only the three fields that contain pointers. The first two fields are pointers to the left and right subtrees; the third is to the null-terminated symbol string. Once you’ve called ClearMemoryRefs, you traverse the tree and note every pointer stored in it. I’ve isolated this code in one debug-only function:

void NoteSymbolRefs(symbol *psym)

{

if (psym != NULL)

{

/*  Validate current node before going deeper  */

NoteMemoryRef(psym);

NoteMemoryRef(psym->strName);

/*  Now do the sub-trees, using recursion  */

NoteSymbolRefs(psym->psymRight);

NoteSymbolRefs(psym->psymLeft);

}

}

This code traverses the symbol table in a pre-order fashion to note every pointer in the tree. Normally, because the symbol table is stored as an in-order tree, I would traverse it in an in-order fashion, but I didn’t do that here because I wanted to validate psym before I dereferenced it. That required a pre-order search. If you do an in-order or a post-order traversal, you must dereference psym before you validate it, and that could lead to a crash, probably after the function wildly recurses many times. True, you would see the bug, but it’s much easier to track down a controlled assertion failure than it is to isolate a random crash.

Once you’ve written Note-Ref routines for your other data structures, you would simply wrap them up in a single routine that you can call from anywhere in your program:

void CheckMemoryIntegrity(void)

{

/*  Mark all blocks as "unreferenced."  */

ClearMemoryRefs();

/*  Note all known global allocations.  */

NoteSymbolRefs(psymRoot);

NoteMacroRefs();

⋮

NoteCacheRefs();

NoteVariableRefs();

/*  Make sure everything is OK.  */

CheckMemoryRefs();

}

The only remaining question is, When do you call this routine? Obviously, you want to call it as often as you can, but when you call it really depends on your program. At a minimum you should call the routine any time you’re about to use the subsystem. Even better, you should check the subsystem any time your program is burning cycles waiting for the user to press a key, move a mouse, or twiddle a hardware switch. You might as well use the wasted time to check things out.

Create thorough subsystem checks, and use them often.

IT’S OBVIOUS ONCE YOU SEE IT

In his book Influence: How and Why People Agree To Things (Morrow, 1984), Dr. Robert Cialdini points out that if you’re a salesperson and somebody walks into your men’s clothing store looking for sweater and a suit, you should always show the person the suits first, and then the sweaters. You’ll make larger sales because after you sell somebody a $500 suit, an $80 sweater is going to look inexpensive by comparison. But if you drag the person to the sweaters first, $80 is going to look outrageous and you’ll probably sell a $35 sweater instead. This is obvious to anybody who takes 30 seconds to think about it, but how many people do?

In the same way, some programmers might have thought that choosing a value for bGarbage was trivial—pick any old number. Other programmers might have thought it unimportant whether you recursed over the symbol table’s tree structure with a pre-order, in-order, or post-order traversal. But, as I pointed out earlier, some choices are better than others.

If you find yourself making an arbitrary choice about an implementation detail, stop and take 30 seconds to review the possibilities; for each one, ask yourself, “Will this cause bugs or will it help find them?” If you asked that question about the possible values for bGarbage, you could see that choosing 0 could cause bugs and that a value such as 0xA3 could help find them.

Choose design details with thought.

THERE’S NO NEED TO KNOW

No doubt you’ll also run across designs for subsystem tests that require various levels of knowledge about the tests in order to use them. Using fValidPointer is an example of this; you can’t use it if you don’t know that it exists. But the best tests are transparent—they work regardless of whether the programmer is aware of them.

Suppose an inexperienced programmer or somebody unfamiliar with your project joins your team. Can’t he or she freely use fNewMemory, fResizeMemory, and FreeMemory without ever knowing about the underlying tests?

What if the new programmer is unaware that fResizeMemory can move blocks and introduces a bug like the one in the assembler I mentioned earlier. Does he or she need to know anything about the integrity checks for those checks to kick in and fire off an “illegal pointer” assertion?

Suppose the new programmer creates a lost memory block? Again, the checks kick in and alert him or her to the problem with a “lost memory” assertion. The new programmer may not even know what a lost memory block is; he or she doesn’t need to know that for the checks to work. Even better, by tracking down the failure, the new programmer will learn about lost memory—without stealing time from an experienced programmer.

This is the power of well-designed subsystem tests—whenever they corner a bug, they grab it by the antennae, drag it to the broadcast studio, and interrupt your regularly scheduled program. You can’t ask for better feedback than that.

Strive to implement transparent integrity checks.

NOT ALL DESIGNS ARE EQUAL

Whenever I’m about to write a new subsystem, I sketch out all the possible implementations, tinkering with details here and there, weighing the pros and cons of each design. Once I determine the best design, only then do I start writing code. I’m sure that’s exactly what you do—it’s what all experienced programmers do.

What I’d encourage you to start doing—if you’re not doing it already—is to include in that design choice an evaluation of each design for how bug-prone the design is and how easy it will be to validate its use by other programmers.

Do the interfaces lend themselves to strong argument validation using assertions? Does the particular design have undefined behavior? If so, can you eliminate that behavior from the design—and if not—does the design make it easy to detect unintentional use of that undefined behavior using assertions or consistency checks? Does the design allow you to incorporate robust and automated debug checks similar to those that I’ve made to the memory manager in this chapter?

Historically, programmers have chosen their designs based on size and speed tradeoffs, on system security issues, and other important considerations. But rarely do they consider how risky their designs are or whether they can validate their designs using debug-only run-time checks.

I’ve chucked many good designs over the years for the only reason that those otherwise good designs made it difficult to detect the obvious bugs that programmers would clearly introduce using the subsystem. I encourage you to consider doing the same.

I’ll be covering specific examples of this principle in Chapter 5.

Choose designs that help you write solid, bug-free code.

A PARADIGM SHIFT

I’ve added a lot of debug code to C’s memory manager in this chapter. Some programmers might even be thinking, “This stuff seems worthwhile, but adding all these checks and including the code for log information is just too much.”

I have to admit that I once felt the same way.

I had a gut revulsion to adding so much inefficiency to a program, but I soon learned that I was wrong. Sure, debug code slows down performance, but what’s worse: having your retail product crash on your users, or having your internal debug version be somewhat slower as it helps you find bugs? You shouldn’t worry too much about the efficiency of debug code. After all, your customers aren’t going to be using the debug version of your code unless you choose to use the crowdsourcing model I talked about in the last chapter.

It’s important to distinguish, at the emotional level, between debug and ship versions of your program. You use the debug version to find bugs. You use the ship version to please customers. And because that’s true, the coding tradeoffs you make for the two versions are radically different.

As long as your product meets the size and speed needs of your customers, you can do anything you want to in your debug code. If adding log routines to the memory manager helps you find all sorts of nasty bugs, everybody wins. Your users have a zippy program, and you find bugs without expending much time and energy.

Years ago when I worked on the Microsoft Excel team, we routinely loaded subsystems with debug code. The Excel code contained memory subsystem tests which were even more thorough than those presented in this chapter. The code also had a thorough cell table integrity check, artificial memory failure mechanisms so that testers could force code to execute out-of-memory error handlers, and a host other checks. That’s not to say that Excel never shipped with bugs during the time I worked on the project—it did—but almost never in code that contained thorough subsystem checks.

It may seem that I’ve added a lot of code around C’s memory manager in this chapter, but consider this: All of the new code was built into the functions fNewMemory, FreeMemory, and fResizeMemory. Nothing was added to the callers of these functions, nor was anything added to the code that implements malloc, free, and realloc.

Even the speed degradation isn’t as bad as you might expect it to be. If your application is packed with assertions and subsystem tests, it should run at about half the speed of your ship version. This is why, with computers running at blistering speeds today, that we’re beginning to see programmers choosing to ship with all debug code intact. After all, half the speed of blisteringly fast is still blisteringly fast.

Don’t apply ship version constraints to the debug version. Trade size and speed for error detection.

FIND BUGS NOW, OR FIND THEM LATER

In this chapter, I’ve covered a half dozen ways to enhance C’s memory management subsystem to be much more robust in terms of writing bug-free code. But remember, these coding philosophies apply to any subsystem you may write, regardless of which language you’re coding in.

Just be aware, though, that when you add DEBUG code and assertions to a subsystem, expect to see your bug counts climb dramatically. This can alarm people if they’re not prepared for it.

I once enhanced a core code library that many groups at Microsoft relied on for their projects, and along with the enhancements, I loaded the new version with DEBUG code and assertion checks to automatically detect unknown bugs. Prior to that, this particular library had never had DEBUG code nor assertion checks.

When I released the library to the development groups, I got an unexpected surprise. One project lead was furious and demanded that I give him a version of the enhanced library without the assertion checks.

I asked him why.

“We installed the library, and we can’t even launch our program without getting one assertion failure after another,” he said.

I was shocked because I had rigorously tested the library before releasing it to the development teams.

“Have you tracked down any of those assertion failures?” I asked.

“We have, and they were bugs in our code. But there are so many assertion failures that they can't all be valid. We don't have time to waste tracking down phantom problems. I need the enhanced library, but without the assertion checks.”

Well, I didn’t think he was seeing phantom bugs, so I asked him to continue to use the library until he found an invalid assertion failure.

He was upset, but he agreed to that, and he never contacted me again about the library.

That project lead panicked because I hadn’t told anybody that I had incorporated thorough DEBUG code and assertion checks into the newly released library. Frankly, it never occurred to me to mention it because the projects using this particular library routinely used DEBUG code and assertion checks in their own code.

Consider this…

How much harder it would be for bugs to go unnoticed in your projects if every subsystem thoroughly validated itself, where bugs were detected automatically, without luck or skill.

That’s how you write rock-solid, bug-free code.

QUICK REVIEW

◆

Look at your subsystems and ask yourself how programmers are likely to misuse them. Add assertions and validation checks to catch hard-to-spot and common bugs.

◆

You can’t fix bugs if you can’t repeatably find them. Look for anything that can cause random behavior and remove it from the debug versions of your subsystems. In this chapter we set “undefined” memory to a constant garbage value. Randomly moving blocks around is another example. Look for similar techniques in the subsystems that you create.

◆

If your subsystems release memory (or other resources) and create “garbage,” scramble the data so it really looks like garbage; otherwise code somewhere may continue to use the data without your noticing it.

◆

Similarly, if your subsystems contain behavior that may happen, but that behavior happens rarely, add debug code to make sure that such behavior does happen, and often. You will increase your odds of catching bugs in code that is not normally executed.

◆

Make sure your debug tests work even for programmers who are unaware of them. The best tests are those that require no knowledge of their existence.

◆

For each design you consider, ask yourself, “How can I thoroughly validate this implementation?” If you find that it would be difficult or impossible to test the proposed implementation, seriously consider a different design, even if that means trading size or speed for the ability to test the system.

◆

Think twice before throwing out a validation test because it would be too slow or take too much memory. Remember, the validation code will be stripped from the ship version of your program. If you find yourself thinking, “This test is too slow (or too big),” stop and ask yourself, “How can I keep this test, but make it faster (or smaller)?”

THINGS TO THINK ABOUT

1.

If, while testing your code, you run across data made up of some combination of 0xA3’s, you know that you’re probably looking at uninitialized data, or at data that has been released. How could you change the debug code to make it easier to determine which type of data you’ve found?

2.

Programmers occasionally write code that fills past the end of an allocated memory block. Describe how you could enhance the memory subsystem checks to alert you to these types of bugs.

3.

Although the CheckMemoryIntegrity routine will catch dangling pointers, there are times that it can’t. For instance, suppose you have a function that calls FreeMemory but a bug in the code leaves a dangling pointer to the free block. Now further suppose that before the pointer can be validated, something calls fNewMemory and reallocates the same block of memory that was released a moment ago. What you’re now left with is a dangling pointer that points to a valid-looking, but different, memory block. That’s a bug, but to CheckMemoryIntegrity, everything looks quite legal. If this were a common bug in your program, how could you enhance the system to detect this problem?

4.

With the NoteMemoryRef routine, you can validate every pointer in your program, but how do you validate the block sizes? For instance, suppose that your code thinks it has a valid pointer to an 18-byte block, but the allocated memory block is shorter than that? Or, what about the reverse case, in which your code thinks it has a 15-byte block, but the log information shows that you’ve allocated 18 bytes? That is often just as bad. How could you strengthen the integrity checks to catch these problems?

5.

The NoteMemoryRef routine in Appendix B lets you mark a block as being referenced, but it doesn’t alert you to a problem if a block is referenced five times when it should be referenced only once. For example, a doubly linked list would have two references to each node: one for the forward pointer, and one for the back pointer. But in most cases, your blocks should have exactly one reference to them and if there are more, you have a bug someplace. How could you improve the integrity checks to allow multiple references to some blocks, but still assert for those for which this should never happen?

6.

Throughout this chapter, I’ve shown many examples of debug code that you could add to C’s memory subsystem to help programmers detect problems, but what about code that you could add to help your Testers detect bugs? Testers know that programmers often mishandle resource allocation error conditions such as out-of-memory or out-of-storage conditions, so how could you give testers the ability to fake such out-of-resource conditions for a particular subsystem?

PROJECT: Look at the major subsystems in your own project. What kinds of debug checks could you implement to catch the common bugs associated with using those subsystems?

4

STEP THROUGH

YOUR CODE

I’ve said before that the best way to find bugs is to execute the code and then somehow spot them, either by eye or by using automated tests such as assertions and subsystem integrity checks. But while assertions and subsystem checks are valuable, they don’t protect you from problems that you haven’t thought of in advance; in that respect, they are like the security system in your home.

If you wire your doors and windows but thieves get in through a skylight or basement opening, the alarm won’t go off. If you put disturbance-sensors on your gaming console, monster-sized TV, surround sound system, and other valuables that you would expect thieves to take, but instead they grab your Barry Manilow collection, the china that you never use, or heaven forbid your NASCAR collector cards, the thieves are going to get away unnoticed.

Similarly, if you validate your function arguments using assertions that set off alarms, but the bugs show up in the logic of your code, the alarms in your assertions will remain silent.

In theory, you could put so many assertions and so much debug code into your programs that no bug could go unnoticed for long before you were alerted to its presence. That’s in theory. In reality, including that much debug code would probably be a waste of time for most projects, and it would still require that you predetermine what the likely bugs are.

Rather than going totally overboard with assertions and debug checks, a better approach is to actively look for bugs when they’re most likely to occur—when you’ve just written new code or changed existing code.

The best way to write bug-free code is to actively step through all new code or modified code in your debugger to watch it execute, to verify that every instruction does exactly what you intended it to do.

In this chapter, I’ll talk not only about why it’s important to step through your code, but about how to do it effectively.

GAIN CONFIDENCE IN YOUR CODE

One time, I was working on a feature for Microsoft’s homegrown Macintosh development system. When I began testing the code, I found a bug and traced it to some new code written by another programmer. What puzzled me about this bug was that it was so central to the other programmer’s code that I couldn’t see how his feature could possibly have worked, so I asked him about it.

“I think I’ve found a bug in the code you just finished.” I said. “Do you have a moment to take a quick look at it?”

He loaded the code into an editor and I showed him where I thought the problem was. When he saw the code, he was surprised.

“You’re right; the code is definitely wrong. I wonder why my tests didn’t catch this bug…”

As the project lead, I asked, “How exactly did you test the code?”

He explained his test to me, and it seemed as if it should have caught the bug. We were both confused.

“Let’s set a breakpoint on the function and step through the code to see what’s really going on.” I suggested.

We tried to do just that, but when we set the breakpoint and hit the “run” key, the test ran to completion; it never hit the breakpoint. That’s why the programmer never saw the bug. It didn’t take much longer to determine why the test wasn’t reaching the breakpoint—a function a few steps up the call chain had an optimization that allowed it to sometimes skip unnecessary work. In this case, it skipped the new code.

Do you remember what I said in Chapter 1 about the problems with black box testing? I said that testers throw inputs at code and decide whether the code works by looking at the outputs—if the outputs look correct, the code must then be working correctly. The problem with that approach is that you can’t tell what goes on between stuffing in the inputs and receiving the outputs.

This programmer missed the bug in question because he tested the code as a black box; he put in some inputs, got the correct outputs, and judged the code to be correct. He didn’t use the extra tools available to him as a programmer.

Programmers, unlike most testers, have the ability to set breakpoints in code, step through code, and watch the process of inputs transforming into outputs. Strangely, though, few programmers make it a habit to step through their code when they test it. Many don’t even bother to set a breakpoint on the code to make sure that the code is executed.

Let’s go back to a point I made in the introduction to this chapter: The best way to catch bugs is to look for them the moment you write or change code. So what is the best way programmers can test their code? It’s by stepping through their code, taking a microscopic look at the intermediate results. I don’t know many programmers who consistently write bug-free code, but the few I do know habitually step through all of their code.

As a project lead, I’ve asked many programmers to walk through their code when they test it. Almost universally, they stare in shock—not because they disagree with the concept but because the process sounds so time consuming. They’re barely staying on schedule as it is… When are they going to find time to step through their code?

Fortunately, that gut reaction is off. Yes, it does take time to step through your code, but only a fraction of the time it takes to write the code. Think about it. When you implement a new function, you must design the interface, figure out the algorithm, and physically type in the lines of program source. How much extra effort does it really take to set a breakpoint when you first run the code, and hit the “step” key as you check each line? Not much, especially once you make it a habit. It’s like learning to drive a stick-shift car—it seems impossible at first, but after a few weeks of driving, you don’t even notice when you shift; you just do it. You can even chew gum while shifting! Similarly, once you make it a habit to step through your code, you don’t think much about setting a breakpoint and going through the process; you just do it. And you catch bugs.

Step through all new code to proactively catch bugs.

FORCING ERRORS IN THE CODE

Of course, there are techniques you can use to make stepping through your code more effective. After all, it doesn’t do much good if you step through your new code but don’t step through all of your new code. For example, every programmer knows that error-handling code often has bugs in it because error-handling code is typically so rarely executed.

So you have a choice here.

Either you can create test cases that force the error conditions to occur, or you can simulate failures while you’re stepping through your code. Simulating failures is much easier and takes much less time, so let’s explore that route. Take a look at this code extract:

pbBlock = (byte *)malloc(32);

if (pbBlock == NULL)

{

handle the error condition;

⋮

}

⋮

When you step through this code, malloc will normally allocate a 32-byte block of memory and return a non-NULL pointer, which causes the code to bypass the error handling code.

To test the error-handling code, set a breakpoint in your debugger on the if statement and re-run the code. When your debugger hits the breakpoint set pbBlock to the NULL pointer right after you see the debugger execute the call to malloc.

malloc may allocate the block, but if you set pbBlock to the NULL pointer, it will look to your code as if malloc failed, allowing you to step through the error handler. (For detail-oriented readers: Yes, malloc’s block will be lost when you change pbBlock, but this is only a test.)

In addition to stepping through your error conditions, you should also step through every possible branch of your code, keeping a keen eye on the data flow to ensure that what you think you coded is what is actually happening.

To be clear, I’m not suggesting that you step through every combination of branches through your code—that could literally be impossible even for small pieces of code. My suggestion is simply to step through each execution branch at least once so that you can watch the data flow.

The obvious cases in which you have more than one code branch in C are the if and switch statements, the ?: operator, and the &&, and || operators that also have two branches because of how they implement short-circuit evaluation. If you’re using a language other than C, there may be other cases where you will have multiple code branches to review.

I can’t tell you how many times I’ve stepped through the condition of an if statement only to be puzzled by what I saw in the debugger. Whenever that happened, a quick review of the source code would typically show that I had mistakenly typed a bitwise & or | operator instead of the logical && or || in the test. Stepping through your code finds these types of bugs quickly and easily.

After you’ve stepped through every branch of your new code, you can be more confident that your code is bug-free—at least you’ll know that all of the code definitely works for some inputs. If you choose good test cases, stepping through your code can be invaluable.

Step through every code branch at least once.

DATA FLOW, THE LIFEBLOOD OF CODE

When I wrote the fast memset routine in Chapter 2, this is how my first version (stripped of assertions) looked:

void *memset(void *pv, byte b, size_t size)

{

byte *pb = (byte *)pv;

if (size >= sizeThreshold)

{

unsigned long l;

/*  Pack a long with 4 bytes. */

l = (b<<24) | (b<<16) | (b<<8) | b;

pb = (byte *)longfill((long *)pb, l, size / 4);

size = size % 4;

}

while (size-- > Ø)

*pb++ = b;

return (pv);

}

That code may look correct, but it contains a subtle bug. After I wrote the code, I ran it in an existing application—no problem, the code worked fine. But to be sure that the code worked, I set a breakpoint on the code and re-ran the application. The moment the code debugger gave me control, I looked at the arguments: The pointer looked valid, so did the size, and the byte was 0. Now I hate testing code using the value 0 because 0 makes it hard to see many types of data bugs, so I immediately changed the byte argument to a weird value like 0x4E.

I first stepped through the case in which size was less than sizeThreshold. That path worked fine. Next I stepped through the case in which size was greater than or equal to sizeThreshold. I wasn’t expecting to have any problems, but when I stepped over this line,

l = (b<<24) | (b<<16) | (b<<8) | b;

I saw that l was set to 0x00004E4E, and not 0x4E4E4E4E, the value I was expecting to see. A quick assembly language dump of the function showed me the bug—and explained why the application worked despite it.

At that time, the C compiler I was using defined integers as 16-bit values, and with 16-bit integers according to C’s type promotion rules, the result of b<<24 is not 0x4E000000 as I expected the code to produce, but is instead 0 because of overflow. The same is true for b<<16. Again 0.

There wasn’t anything wrong with the logic of the code, but the implementation was flawed. The code initially appeared to work in the application I tested because it used memset to zero-fill blocks of memory, and 0<<24 is 0, the correct answer, but for the wrong reason.

I was able to catch that bug almost immediately because I spent an extra minute to step through the code before setting it aside and moving on. True, the bug was serious enough that somebody would have spotted it eventually, but remember, the goal is to catch bugs at the earliest possible moment. Stepping through code helps to achieve that goal.

The real power in stepping through your code is that it lets you see the data as it flows through your logic. If you were to focus on data flow as you stepped through your code, consider how many of these bugs you would catch without much effort:

◆

Overflow and underflow bugs

◆

Data conversion bugs

◆

Off-by-one bugs

◆

NULL pointer bugs

◆

Bugs using garbage memory ( 0xA3 bugs)

◆

Assignment bugs in which you use = instead of ==

◆

Condition bugs in which you use & instead of && or | instead of ||

◆

Precedence bugs

◆

Logic bugs

Wouldn’t you catch all of those bugs? The value in focusing on the data is that it gives you a second, very different view of your code. You may not notice the assignment bug in this code:

if (ch = '\t')

ExpandTab();

but when you step through it focusing on data flow, it’s easy to see ch getting clobbered.

As you step through code, focus on the data flow.

ARE YOU MISSING SOMETHING?

One problem with using a source level debugger is that stepping over a line of code can cause you to miss important details. Suppose that instead of typing && in the code below, you had typed & by mistake:

/*  If the symbol exists and it has a textual

*  name, then release the name.

*/

if (psym != NULL  &  psym->strName != NULL)

{

FreeMemory(psym->strName);

psym->strName = NULL;

}

This code is legal, but it’s wrong. The intent of the if statement is to keep a NULL psym pointer from being used to reference the strName field of a symbol structure, but the code doesn’t do that. Instead, the code always references the strName field regardless of whether psym is NULL.

If you use a source level debugger to step through the code and hit the “step” key when you reach the if statement, the debugger will execute the entire line as one operation. But to spot the bug, you would have to see that the right-hand side of the expression is executed even when the left side is FALSE.

Remember what I said earlier: The &&, ||, and ?: operators have two code branches each, and to catch bugs, you must step through both branches. The problem with a source level debugger is that it steps over both branches of the &&, ||, and ?: operators with a single step. There are two practical approaches to overcoming this problem.

First, anytime you step to a compound conditional using && and || operators, scan the list to verify that they are “spelled” correctly, and then use the debugger to display the result for each side of the expression. This will help you catch bugs in which the full expression evaluates correctly, but for the wrong reason. For example, if you think the first part of an || expression should be TRUE and the second part FALSE but just the opposite is so, the expression will incorrectly evaluate the correct result. Looking at the individual parts of the expression will alert you to such problems.

A second, more thorough approach is to step through compound conditionals and ?: operations at the assembly language level. Yes, this takes extra effort, but for critical code it is important that you actually step through the code to see the intermediate results. As with stepping through your code at the source level, stepping through it in assembly language is quick once you’re used to doing it; it just takes practice.

Now for the fun part…

If you’re using a good optimizing compiler, stepping through your code can be an interesting exercise because the compiler may intermix the machine code of adjacent source lines as it tries to generate optimal code. It’s not at all uncommon for one “step” command to step over three lines of source code, nor is it unusual to step across a source line that moves data from one spot to another and find that the data has not moved yet because the code optimizer found a later, more efficient, time to move the data.

To make it easier to step through your code, consider turning off unnecessary compiler optimizations in the debug version of your program; those optimizations do nothing but scramble your code.

I’ve heard programmers argue that by disabling optimizations, you introduce risk because you create unnecessary differences between the debug and ship versions of the code. There is truth in that reservation, especially if you’re concerned about code-generation bugs that may be in your compiler. But remember that the purpose of the debug version is to catch bugs and if disabling optimizations helps you to do that, it’s worth considering.

The best approach is to try stepping through your optimized code to see how difficult it is. If you find that you must disable optimizations to step through the code effectively, then do it. You may miss the rare compiler generation bug, but you’ll find many of your own bugs. The trade-off is worthwhile.

Source level debuggers can hide execution details. Step through critical code at the instruction level.

TRY IT—YOU’LL LIKE IT

I wish I knew a way to persuade programmers to step through their code, or at least to get them to try it for a month. But I’ve found that programmers in general can’t get over thinking it will take too much time. That’s one of the advantages of being a project lead; you can be a bit autocratic and insist that programmers on your team step through their code until they learn that it doesn’t take much time and that it is worthwhile.

If you’re not already stepping through your code, will you start to? Only you know the answer. But I’m guessing that you picked up this book and began reading because you’re serious about reducing the number of bugs in your code or in the code of the programmers you lead. It really comes down to this choice: Would you rather spend a small amount of time up front verifying your code by stepping through it, or would you rather let bugs get into the master sources and hope that testers will notice them so that you can fix them later? The choice is yours.

QUICK REVIEW

◆

Bugs don’t grow in code spontaneously; they are the result of a programmer’s writing new code or changing existing code. If you want to find bugs in your code, there is no better method than stepping through every line of the code the first time you run it, checking data flow through each code branch.

◆

Although your gut reaction might be that walking through your code will take a lot of time, your gut reaction would be wrong. Yes, initially it will take more time—until walking through your code becomes habitual. Once that happens, you’ll zip right through it.

◆

Be sure to step through every code branch—especially in your error handling code—at least once. Don’t forget that the &&, ||, and ?: operators have two code branches to test.

◆

In some cases, you may need to step through code at the assembly language level. While you don’t need to do this often, don’t avoid doing it when it’s necessary.

PROJECT: Take a look at the bugs that have been reported against your code over the last six months. How many would you have caught had you stepped through the code in your debugger at the time you wrote it?

5

CANDY-MACHINE

INTERFACES

When I worked at Microsoft, one of the perks that Microsoft gave its employees was free soft drinks, flavored seltzer water, milk (chocolate too!), and those little cartons of fruit juice. As much as you wanted. But, darn it, if you wanted candy, you had to pay for that yourself.

Occasionally, I would get the munchies and stroll down to a vending machine. I’d plunk in my quarters, press 4 and then 5 on the selection keypad, and watch in horror as the machine spat out jalapeño-flavored bubble gum instead of the Grandma’s Peanut Butter Cookie I thought I had asked for. Of course, the machine was right and I was wrong—number 45 was the gum. A quick look at the little sign by the cookie would always verify my mistake: No. 21, 45¢.

That candy machine always irritated me because if the engineers had spent an extra 30 seconds thinking about their design, they could have saved me, and I’m sure countless others, from getting something they didn’t want.

The engineers only needed to ask one simple question during the design phase of the vending machine: “How might users punch in the wrong code by mistake?” They could have easily seen that people might still have the price in mind as they punched in their selection. To prevent that mistake, I’m sure those engineers, had they asked themselves the question, would have seen that using an alphabetic selection code instead of a numeric one would have prevented any mistakes: Press AE, 45¢.

The vending machine wouldn’t have cost one extra dime to make, and the improvement wouldn’t have changed the design in any appreciable way, but every time someone turned to the keypad and tried to punch in 45 (the price, not the code), they would be confronted with an alphabetic keypad. The user would, in effect, be reminded to do a double-check and punch in the correct letter code instead of the price that they were holding in their mind. That interface design would have led people—especially me—to enter the right code.

When you design function interfaces in your code, you face similar problems. Programmers aren’t often trained to think about how other programmers will use their functions, but as with the candy machine, a trivial difference in design can either cause bugs or prevent them. It’s not enough that your functions be bug-free; they must also be safe to use. Taking a moment to ask the question, “How might callers to this function make mistakes using it?” and then tweaking your interface to eliminate the problems areas is a question well worth asking.

getchar GETS AN int, OF COURSE

Many of the standard C library functions, and thousands of functions patterned after them, have candy-machine interfaces that can trip you up. Think about the getchar function, for instance. The interface for getchar is risky for several reasons, but the most severe problem is that its design encourages programmers to write buggy code. Consider what Brian Kernighan and Dennis Ritchie have to say about the getchar function in The C Programming Language:

Consider the code

char c;

c = getchar();

if (c == EOF)

⋮

On a machine which does not do sign extension, c is always positive because it is a char, yet EOF is negative. As a result, the test always fails. To avoid this, we have been careful to use int instead of char for any variable which holds a value returned by getchar.

With a name such as getchar it’s natural to define c to be a character, and that’s why this bug so often catches programmers. But really, is there any reason getchar should be so hazardous? It’s not doing anything complex; it’s simply trying to read a character from a device and returning a possible error condition.

The code below shows another problem common in many function interfaces:

/*  strdup -- allocate a duplicate of a string. */

char *strdup(char *str)

{

char *strNew;

strNew = (char *)malloc(strlen(str)+1);

strcpy(strNew, str);

return (strNew);

}

This code will work fine until you run out of memory and malloc fails, returning NULL instead of a pointer to memory. Who knows what strcpy will do when the destination pointer, strNew, is NULL, but whether strcpy crashes, or quietly trashes memory, the result won’t be what you intended.

Programmers have trouble using getchar and malloc because they can write code that appears to work even though it’s flawed. It’s not until weeks or months later that the code crashes unexpectedly because, as in the sinking of the Titanic , a precise series of improbable events takes place and leads to disaster. Neither getchar nor malloc lead programmers to write correct code; both lead programmers to ignore the error condition.

The problem with getchar and malloc is that their return values are imprecise. Sometimes they return the valid data that you expect, but other times they return magic error values.

If getchar didn’t return the funny EOF value, declaring c to be a character would be correct and programmers wouldn’t run into the bug that Kernighan and Ritchie talk about. Similarly, if malloc didn’t return NULL as though it were a pointer to memory, programmers wouldn’t forget to handle the error condition. The problem with these functions is not that they return errors, but that they bury those errors in normal return values where it’s easy for programmers to overlook them.

So consider this…

What if you redesigned getchar so that it returned both outputs separately? It could return TRUE or FALSE depending upon whether it successfully read a new character, and the character itself could be returned in a variable that you pass by reference:

flag fGetChar(char *pch);    /*  prototype  */

With the interface above, it would be natural to write

char ch;

if (fGetChar(&ch))

/* ch has the next character */

else

/* hit EOF, ch is garbage */

The problem with char vs. int goes away, and it’s unlikely that any programmer, no matter how green, would accidentally forget to test the error condition.

Compare the return values for getchar and fGetChar. Do you see that getchar emphasizes the character being returned, whereas fGetChar emphasizes the error condition? Where do you think the emphasis should be if your goal is to write bug-free code?

True, you do lose the flexibility to write code such as

putchar(getchar());

but how often are you certain that getchar won’t fail? In almost all cases, the code above would be wrong.

Some programmers might think, “Sure, fGetChar may be a safer interface, but you waste code because you have to pass an extra argument when you call it. And what if a programmer passes ch instead of &ch? After all, forgetting the & is an age-old source of bugs when programmers use the scanf function.”

Good questions.

Whether the compiler will generate slightly better or worse code is actually compiler dependent. But unless you’re working with embedded systems or on hardware in which every byte is precious and every cycle matters, such a miniscule difference is not going to be noticed, particularly on a modern desktop system that has blazing multi-core processors, gigabytes of RAM, and terabytes of disk space. The days of micro-tuning code are long behind us.

The second concern—passing a character to fGetChar instead of a pointer to a character—shouldn’t worry you if you’re consistently declaring prototypes for all your functions. That’s just good programming practice in C and C-derived languages. If you pass fGetChar anything but a pointer to a character, the compiler will automatically generate an error and show you your mistake.

The reality is that combining mutually exclusive outputs into a single return value is a carryover from assembly language programming where you have a limited number of machine registers to manipulate and pass data. In that environment, using a single register to return two mutually exclusive values is not only efficient but often necessary. Coding in C and other modern high-level languages is another matter—even though C lets you “get close to the machine,” that doesn’t mean you should write high-level assembly language.

When you design your function interfaces, choose designs that lead programmers to write correct code the first time. Don’t use confusing dual-purpose return values—each output should represent exactly one data type. Make it hard to ignore important details by making them explicit in the design.

Make it hard to ignore error conditions.

Don’t bury error codes in other data.

JUST A LITTLE EXTRA THOUGHT

Programmers know when they’re combining multiple outputs into a single return value, so acting on the previous suggestion is easy—they just stop doing it. In other cases, though, an interface can seem fine, but like the Trojan horse, contain hidden danger. Take a look at this code to change the size of a memory block:

pbBuf = (byte *)realloc(pbBuf, sizeNew);

if (pbBuf != NULL) {

/*  use/initialize the larger buffer */

}

Do you see what’s wrong with this? If you don’t, don’t worry—the bug is serious, but it’s subtle, and very few people spot it unless they’re given a hint. So here’s a hint: If pbBuf is the only pointer to the block about to be resized, what happens if the call to realloc fails? The answer is that NULL is stuffed into pbBuf when realloc returns, destroying the only pointer to the original block.

The code creates lost memory blocks.

Here’s a question: How many times do you want to resize a block and store the pointer to the resized block in a different variable? I’d imagine about as often as you’d want to drive to a restaurant in one car and leave in another. Sure, there are cases in which you want to store the new pointer in a different variable, but normally, if you change the size of a block, you want to update the original pointer. That’s why programmers so often fall into realloc’s trap. realloc has a candy-machine interface.

Ideally, realloc would always return an error code and a pointer to the memory block regardless of whether the block was expanded. That’s two separate outputs. Let’s take another look at fResizeMemory, the wrapper function for realloc that I talked about in Chapter 3. Here it is again, stripped of all the debug code:

flag fResizeMemory(void **ppv, size_t sizeNew)

{

byte **ppb = (byte **)ppv;

byte *pbNew;

pbNew = (byte *)realloc(*ppb, sizeNew);

if (pbNew != NULL)

*ppb = pbNew;

return (pbNew != NULL);

}

Take a look at the if statement in the code above—it ensures that the original pointer is never destroyed. If you rewrote the realloc code at the start of this section using fResizeMemory, you would have:

if (fResizeMemory(&pbBuf, sizeNew))

/*  use/initialize the larger buffer */

In this case, if the attempt to resize the block fails, pbBuf is left untouched and continues to point to the original block; pbBuf is not set to NULL. This behavior is exactly what you want. So here’s a question: “How likely is it that a programmer will lose a memory block using fResizeMemory?” Here’s another: “How likely is it that a programmer will forget to handle fResizeMemory’s error condition?” In both cases, not likely.

Another interesting point to note is that programmers who habitually follow the earlier suggestion in this chapter—“Don’t bury error codes in return values”—would never design an interface such as realloc’s. Their first attempt would be more like fResizeMemory’s—and so wouldn’t have realloc’s “lost block” problem. The recommendations in this book build on each other and interact in ways that you might not expect. This is an example of that happening.

Separating your outputs is not always going to prevent you from designing interfaces with hidden traps. I wish I could offer a better piece of advice, but the only sure way to catch such hidden traps is to stop and think about your designs. The best approach is to examine every possible combination of inputs and outputs and look for side effects that can cause problems. I know this can sometimes be tedious, but remember, it’s relatively cheap for you to take the extra time up front to think about your design. The worst thing you can do is skip this step and force who knows how many other programmers into tracking down and fixing bugs caused by a candy-machine interface.

Imagine how much total time has been wasted by programmers all over the world who have been forced to track down bugs caused by the interface traps of getchar, malloc, and realloc —to say nothing of all the functions that have been written using one of these three as a model. It’s a sobering amount of time.

I don’t mean to suggest that the programmers who designed the standard C library didn’t know what they were doing. In fact, those programmers were at the top of their game back then. The C library was designed in the early 1970s when computer science was still a fledgling field, with a tremendous learning curve ahead of it on how to write large-scale, maintainable, bug-free code.

Recall that when the C library was first designed and written, the practice of indenting your if and while statements was a radically new idea that most programmers dismissed out of hand. FORTRAN was king-of-the-hill, and FORTRAN code always started in column 1. Who needs that crazy indenting stuff? Today, you’d be the first programmer voted off the island if you wrote every line of code starting in column 1. Development practices evolve with experience…

Always look for, and eliminate, flaws in your interfaces.

THE ONE-FUNCTION MEMORY MANAGER

Although I spent a lot of time talking about the realloc function in Chapter 3, I didn’t cover many of its more bizarre aspects. If you pull out your C library manual and look up the full description of realloc, you’ll find something like this:

void *realloc(void *pv, size_t size);

realloc changes the size of a previously allocated memory block. The contents of the block are preserved up to the lesser of the new and old block sizes.

◆

When resizing a memory block, realloc may physically move the memory block to accommodate the request, returning a pointer to the location of the resized memory block. If the memory block is being expanded, the contents of the expanded part of the memory block will be uninitialized.

◆

If you attempt to resize a memory block and realloc cannot satisfy the request, NULL is returned.

◆

If pv is NULL, realloc behaves as though you had called malloc(size) and returns a pointer to a newly allocated memory block, or NULL if the request cannot be satisfied.

◆

If the new size is 0 and pv is not NULL, realloc behaves as though you had called free(pv) and NULL is always returned.

◆

If pv is NULL and size is zero, the result is undefined.

Whew! realloc is a prime example of implementation overkill—it’s a complete memory manager in just one function. Why do you need malloc? Why do you need free? realloc does it all.

There are several good reasons why experienced software developers have learned over the years to never design functions like realloc.

First, you can’t expect programmers to use such a function safely. There are so many details that even experienced programmers don’t know them all. If you doubt this, take a survey and tally how many programmers know that passing a NULL pointer to realloc simulates a call to malloc. Tally how many know that passing a 0 size is the same as calling free. True, this is fairly arcane behavior, so ask them a question they must know the answer to if they hope to avoid bugs. Ask them what happens when they call realloc to expand or shrink a block. Do they know that the block can move?

Here’s another problem with realloc: We know it’s possible to pass garbage to realloc, but because its definition is so general it’s hard to guard against invalid arguments. If you pass a NULL pointer to realloc by mistake, that’s legal. If you pass a 0 size by mistake, that’s legal too. It’s too bad if you malloc a new block or free the current one when your intent is to resize a block. How can you assert that realloc’s arguments are valid if practically everything is legal? No matter what you throw at it, realloc handles it, even to extremes. At one extreme it frees blocks; at the other it mallocs them. These are totally opposite behaviors.

To be fair, programmers don’t usually sit down and think, “I’m going to design an entire subsystem in a single function.” realloc and functions like it almost always arise for one of two reasons. Either they evolve into multipurpose functions, or the extra behavior (such as free and malloc) falls out of the implementation and the programmer extends the formal description to include this “fortunate” behavior.

If, for whatever reason, you write a multipurpose function, break it down into its distinct behaviors. For realloc the breakdown would be expanding a block, shrinking a block, allocating a block, and freeing a block. By breaking realloc down into four distinct functions, you’ll be able to do much better error checking. If you’re shrinking memory, for instance, you’ll know that the pointer must be to a valid block and you’ll know the new size must be less than (or possibly equal to) the current size. Anything else is an error. With a separate fShrinkMemory function, you could use assertions to validate those arguments.

In some cases, you may actually want a function to do more than one task. For example, when you call realloc, do you usually know whether the new size will be smaller or larger than the current one? Whether you know depends on your program, but I usually don’t know (although I can often derive the information). I’ve found it better to have one function that both shrinks and expands blocks so that I don’t have to write if constructs every time I need to resize memory. True, I give up some extra argument checking, but this is offset by the ifs that I no longer need to write (and possibly mess up). I do always know when I’m allocating memory or freeing it, so I would rip those tasks out of realloc and make them separate functions. The fNewMemory, FreeMemory, and fResizeMemory from Chapter 3 are these three well-defined functions. I would include two more functions to round out the set:

flag fExpandMemory(void **ppv, size_t sizeLarger);

flag fShrinkMemory(void **ppv, size_t sizeSmaller);

I would create these two functions specifically so that I could include assertions in both functions to verify my intentions. If I knew I was expanding a memory block and I called fExpandMemory with arguments that were telling it to actually shrink the block, I would want to know that. If I called fShrinkMemory with values that asked it to expand the block, I would absolutely want to know that as well.

Every function you write should have one, well-defined purpose. This allows you to thoroughly validate the function’s intended purpose.

Write single-purpose functions to allow for stronger argument validation.

WISHY-WASHY INPUTS

Earlier I said that your outputs should be separate and explicit to avoid confusing the programmers who use your functions. If you also apply that advice to the inputs of your functions, you naturally avoid writing all-encompassing functions such as realloc. realloc takes a pointer to a memory block, but sometimes it can be the magic NULL value that forces realloc to mimic malloc. realloc also takes a size value, but it allows the magic size of 0 that forces it to mimic free. These magic numbers may seem harmless enough, but they destroy comprehension. Quick, is the code below resizing, allocating, or releasing a memory block?

pbNew = realloc(pb, size);

You can’t tell; it could be doing any one of those actions—it all depends on the values of pb and size. Just as explicit outputs make it easier to decipher what’s going on, so do explicit inputs, and that explicitness can be invaluable to maintenance programmers who have to read and understand code they didn’t write.

Sometimes wishy-washy inputs aren’t as easy to spot as realloc’s. For example, take a look at this specialized string copy routine that takes the first size characters of strFrom and turns them into a string stored at strTo:

char *CopySubStr(char *strTo, char *strFrom, size_t size)

{

char *strStart = strTo;

while (size-- > Ø)

*strTo++ = *strFrom++;

*strTo = '\Ø';

return(strStart);

}

CopySubStr is similar to C’s standard strncpy function, but unlike strncpy, it guarantees that the string at strTo is a true zero-terminated C string. You would typically use CopySubStr to extract a portion of a larger string—say, to pull the name of a day out of a packed string:

static char *strDayNames = "SunMonTueWedThuFriSat";

⋮

ASSERT(day >= Ø  &&  day <= 6);

CopySubStr(strDay, strDayNames + day*3, 3);

Now that you understand how CopySubStr works, do you see the questionable input? It’s easy to spot if you try to write assertions to validate the arguments. The validation for strTo and strFrom would be

ASSERT(strTo != NULL  &&  strFrom != NULL);

but how do you validate the size argument? Is 0 a legal size? What if the size is larger than the length of strFrom? If you look at the code, you’ll see that it handles both cases, sort of. The while loop quits if size is 0 on entry, so that works, and if size is larger than strFrom, the while loop will copy the entire string, including the null character terminating it. You just need to document the function to explain it:

/*  CopySubStr -- extract a substring from a string.

*

*  Convert the first "size" characters of strFrom into

*  a string stored at strTo. If there are fewer than

*  "size" characters in strFrom, then the entire string

*  is copied to strTo. If size is Ø, strTo is set to

*  the empty string.

*/

char *CopySubStr(char *strTo, char *strFrom, size_t size)

{

⋮

Sound familiar? Sure, because functions that behave that way are as common as dust on light bulbs. Still, is this the best way to handle the size input? The answer is No, at least not when you view it in terms of writing solid, bug-free code.

Suppose, for example, that a programmer had mistyped 3 as 33 when he or she wrote the code to call CopySubStr:

CopySubStr(strDay, strDayNames + day*3, 33);

That’s a realistic bug, but given its definition it’s perfectly legal to call CopySubStr with a ridiculous value like 33. Oh sure, you would probably catch the bug before you released the code, but you wouldn’t find it automatically the very first time someone called the function with questionable arguments.

From a “bug-free” point of view, if an argument is out-of-range or is meaningless, it should be illegal, because by silently accepting oddball values, you hide bugs rather than find them. Allowing “loose” inputs is another form of defensive programming. Keep the defensive code for robustness, but disallow the questionable inputs:

/*  CopySubStr -- extract a substring from a string.

*

*  Convert the first "size" characters of strFrom into

*  a string stored at strTo. There must be at least

*  "size" characters in strFrom.

*/

char *CopySubStr(char *strTo, char *strFrom, size_t size)

{

char *strStart = strTo;

ASSERT(strTo != NULL  &&  strFrom != NULL);

ASSERT(size <= strlen(strFrom));

while (size-- > Ø)

*strTo++ = *strFrom++;

*strTo = '\Ø';

return(strStart);

}

Sometimes allowing a meaningless argument—such as a size of 0—is worthwhile because it can eliminate unnecessary tests at the caller. For example, since memset allows its size argument to be 0, you don’t need the if statement in the code below:

if (strlen(str) != Ø)          /*  Fill str with spaces. */

memset(str, ' ', strlen(str));

But be careful when you allow 0 sizes. Programmers regularly handle sizes (or counts) of 0 because they can, not because they should. If you write a function that takes a size, you’re not required to handle 0. Instead, ask yourself, “How often will programmers call this routine with a size of 0?” If the answer is never, don’t handle 0; assert instead. Remember that every time you ease a restriction, you eliminate a chance to catch a corresponding bug. A good rule is to initially choose strict definitions for your inputs to maximize the effectiveness of your assertions. If you later find that a restriction is overly harsh, you can remove it without affecting the rest of your code.

I used this philosophy in Chapter 3 when I included the NULL pointer check in FreeMemory. Since I would never call FreeMemory with a NULL pointer, it was more important to me to have the stronger error checking. Your views may be different—there’s no right and wrong here. Just be sure that whatever you do is a conscious choice and not merely a habit you’ve picked up.

Don't be wishy-washy.

Define explicit function arguments.

DON’T FAIL ME NOW

Microsoft has a policy of asking potential employees technical questions during their interviews, and for programmers, that means being handed one or more coding problems. I would start candidates off with the task of writing the standard tolower function. I would hand the candidate an ASCII table and ask, “How would you write a function that would convert an uppercase letter to its lowercase equivalent?” I would be deliberately vague about how to handle symbols and lowercase letters, primarily to see how the programmer would handle those cases. Would those characters remain unchanged? Would symbols and lowercase letters be ignored? More than half the time, candidates would write something like this:

char tolower(char ch)

{

return (ch + 32);

}

This code works if ch is an uppercase letter, but if ch is anything else, it breaks. When I’d point this out to interviewees, sometimes they’d say, “I assumed that the character had to be uppercase. I could handle all characters by returning the character unchanged if it wasn’t a capital letter.” That’s a reasonable solution; other solutions were less so. More often than not, candidates would say, “I didn’t think of that. I can fix the problem by returning an error if ch isn’t a capital letter.” Sometimes they’d have tolower return NULL; at other times they’d return the null character, but the clear winner, for some reason, was -1:

char tolower(char ch)

{

if (ch >= 'A'  &&  ch <= 'Z')

return (ch + 32);

else

return (-1);

}

Returning -1 violates the interface suggestion I made earlier because it mixes an error value with a real piece of data. The problem here is not that the candidates failed to heed a suggestion that they’d probably never heard of, but rather that they were generating an error condition when they didn’t need to. If the tolower function were to return -1, you couldn’t simply write:

ch = tolower(ch);

Instead, you would have to write something like this:

int chNew;    /*  This *must* be an int to hold -1.  */

if ((chNew = tolower(ch)) != -1)

ch = chNew;

If you consider how you’d have to use tolower at every call, you can see that returning an error may not be the best way to define the function.

The lesson here is fundamental: if you find yourself designing a function that returns an error, stop and ask yourself whether there’s any way you can redefine the function to eliminate the error condition. Instead of defining tolower to “return the lowercase equivalent of an uppercase letter,” have tolower “return the lowercase equivalent of ch if one exists; otherwise, return the character unchanged.”

If you find that you can’t eliminate an error condition, consider disallowing the problematic cases altogether. For example, you could require that the argument to tolower be a capital letter and say that any other character is illegal. You would then use an assertion to verify the argument:

char tolower(char ch)

{

ASSERT(ch >= 'A'  &&  ch <= 'Z');

return (ch + 32);

}

Whether you redefine a function or eliminate the problematic cases, you remove the need for callers to do runtime error checking, which results in smaller code and fewer bugs.

As an aside, I’ve received many messages from readers of the first edition of this book asking me how having candidates code C’s tolower function could, in any way possible, demonstrate their programming skills. Appendix D answers that question.

Write functions that, given valid inputs, cannot fail.

READ BETWEEN THE LINES

I can’t emphasize enough how important it is to examine your interfaces from the point of view of the caller. When you consider that you define a function just once, but you call it from many places, it seems foolish not to examine it from the caller’s viewpoint. The getchar, realloc, and tolower examples we’ve seen bring this point home—all complicate code at the point of call. But fusing outputs and returning needless error codes aren’t the only ways in which you can complicate code. Sometimes all it takes is a careless disregard for how the function will “read” when you call it.

Suppose you were trying to improve the disk handling portion of your application and you ran into a file seek call written this way:

if (fseek(fpDocument, offset, 1) == Ø)

⋮

You can tell that some sort of seek is happening, and you can see that the error is being handled, but how readable is the call? What kind of seek is happening—from the beginning of the file, from the current file position, or from the end-of-file? If the return value is 0, does that indicate success or failure?

Suppose, instead, that the programmer had written the call using pre-defined names:

#include <stdio.h>       /*  Pull in SEEK_CUR. */

#define ERR_NONE Ø

⋮

if (fseek(fpDocument, offset, SEEK_CUR) == ERR_NONE)

⋮

Does this clarify the call? Sure it does. But this isn’t a surprising new revelation—programmers have known for years that they should avoid using magic numbers in their code. Not only do named constants make code more readable, but they also make it more portable.

What I would like to point out is that NULL, TRUE, and FALSE are not named constants the way many programmers use them, but rather textual representations of magic numbers. For instance, what do the calls here do?

UnsignedToStr(u, str, TRUE);

UnsignedToStr(u, str, FALSE);

You can probably guess that these calls convert an unsigned value to its textual representation, but how does the boolean argument affect that conversion? Would it be clearer if I instead wrote the calls as:

#define BASE1Ø 1

#define BASE16 Ø

⋮

UnsignedToStr(u, str, BASE1Ø);

UnsignedToStr(u, str, BASE16);

When programmers sit down to write such functions, the boolean values may seem perfectly clear. First the programmers launch into a description and then into the implementation:

/*  UnsignedToStr

*

*  This converts an unsigned value to its textual

*  representation. If fDecimal is TRUE, u is

*  converted to a decimal representation, otherwise

*  it's converted to a hexadecimal representation.

*/

void UnsignedToStr(unsigned u, char *strResult, flag fDecimal)

{

⋮

What could be clearer than that?

The reality is that boolean arguments often indicate that the designer didn’t put much thought into what he or she was doing. Either the function is doing two different things and the boolean argument selects which of the two behaviors you want, or the function is generally flexible but the programmer used a boolean to specify the only two cases he or she was interested in. Often both are true.

If you view UnsignedToStr as a function that is doing two different things, you could drop the boolean argument and split UnsignedToStr into two specific functions:

void UnsignedToDecStr(unsigned u, char *str);

void UnsignedToHexStr(unsigned u, char *str);

But a better solution—in this case—would be to make UnsignedToStr more flexible by changing the boolean argument to a general purpose one. Instead of passing in TRUE or FALSE, have programmers pass in the conversion base:

void UnsignedToStr(unsigned u, char *str, unsigned base);

This gives you a clean flexible design that makes the calling code understandable and at the same time increases the usefulness of the function.

This advice may seem to contradict what I said earlier about defining your arguments rigidly—we went from a concrete TRUE or FALSE input to a general one where most of the possible values aren’t useful. But remember, base may be general, but you can always include an assertion to verify that base is always either 10 or 16. If you later decide that you also need binary or octal conversions, you can relax the assertion to allow programmers to pass in 2 and 8.

That’s far better than some of the functions I’ve seen that have an argument that takes the values TRUE, FALSE, 2, and -1! Because boolean arguments don’t extend easily you either end up with such nonsense, or you have to update every pre-existing call.

Make the code intelligible at the point of call.

Avoid boolean arguments.

WARN PEOPLE OF THE HAZARDS

As a final guard against bugs, you can help prevent other programmers from misusing your functions by writing your documentation so that it both emphasizes the hazards and shows how you expect people to use the code. For instance, instead of documenting getchar as

/*  getchar -- this is the same as getc(stdin)  */

int getchar(void)

⋮

which doesn’t really help programmers much, you could write something like this:

/*  getchar (equivalent to getc(stdin))

*

*  getchar returns the next character from stdin.  When

*  an error occurs it returns the *int* EOF.  A typical

*  use is:

*

*      int ch;       //  ch *must* be an int to hold EOF

*

*      if ((ch = getchar()) != EOF)

*          //  success -- ch has the next character

*      else

*          //  failure -- ferror(stdin) gives error type

*/

int getchar(void)

⋮

If you were to hand both of these descriptions to a programmer just learning the C library, which one do you think would leave stronger impressions about the hazards of using getchar? And what about when that programmer uses getchar for the first time? Do you think she’s going to make up new code, or do you think she will simply copy the “typical use” example from your documentation and adapt it to her needs?

Another positive side effect of documenting functions this way is that it forces less careful programmers to stop and think about how other programmers have to use their functions. If a programmer writes a function with a clunky interface, he should notice that the interface is bad when he tries to write the “typical use” example. But even if he doesn’t notice the problems with the interface, it won’t matter as long as the example is thorough and correct. For example, what if the documentation for realloc provided a typical use example such as this one:

/*  realloc(pv, size)

*  ...

*  A typical use is:

*

*      void *pvNew;   // Protect pv if realloc fails

*

*      pvNew = realloc(pv, sizeNew);

*      if (pvNew != NULL)

*      {

*          //  success -- update pv

*          pv = pvNew;

*      }

*      else

*          //  failure -- leave pv untouched

*/

void realloc(void *pv, size_t size)

⋮

By copying such an example, less cautious programmers are more likely to avoid the lost memory problem that I talked about earlier in this chapter. Your examples won’t protect all programmers, but like the warnings on medicine bottles, they will influence some people, and every bit helps.

But don’t use examples as a substitute for writing good interfaces. getchar and realloc both have bug-prone interfaces—their hazards should be eliminated, not merely documented.

Write comments that emphasize potential hazards.

THE DEVIL IS IN THE DETAILS

Designing bug-resistant interfaces is not difficult, but it does take some extra thought and a willingness to abandon ingrained coding habits. The suggestions in this chapter show how, with simple changes to your interfaces, you can lead programmers to write correct code without much thought on their part. The key concept running through this chapter is “Make everything as clear and as obvious as possible.” If programmers understood and remembered every detail, they might not make mistakes; but programmers do make mistakes, in part because they forget about or never learned about the important details. Make it hard for programmers to unwittingly ignore details; design bug-resistant interfaces.

QUICK REVIEW

◆

Create function interfaces that are easy to use and understand: Ensure that every input and output represent exactly one type of data. Mixing error and other special-purpose values into your inputs and outputs does nothing but clutter your interfaces.

◆

Design your function interfaces in such a way that programmers are forced to think about all the important details, such as handling error conditions. Don’t make it easy for them to ignore or forget the details.

◆

Consider how programmers must call your functions. Look for flaws in your function interfaces that can cause programmers to unwittingly introduce bugs. Of particular importance: Strive to write functions that always succeed so that callers don’t need to do any error handling.

◆

Increase comprehension and thus reduce bugs by making sure that the calls to your functions are understandable to programmers who have to read those calls. Magic numbers and boolean arguments work against this goal.

◆

Break apart multipurpose functions. Not only do the more specific function names increase comprehension (e.g., fShrinkMemory instead of realloc), but you can use more rigid assertions to automatically detect bad arguments.

◆

Document your interfaces to show programmers how to properly call your functions. Emphasize the danger zones.

THINGS TO THINK ABOUT

1.

The strdup function at the beginning of the chapter allocates a duplicate string, but returns NULL if it fails. What would be a more bug-resistant interface for strdup?

2.

I said that the presence of boolean inputs often indicates that there could be a better interface for the function. But what about boolean outputs? For example, if fGetChar fails, it returns FALSE and requires programmers to call ferror(stdin) to determine the cause of the error. What might be an even better interface for getchar?

3.

Why is the C’s strncpy function bound to trip up unwary programmers?

4.

If you’re familiar with C++’s inline function specifier, describe its contribution to writing bug-resistant interfaces.

5.

C++ introduced & reference arguments similar to Pascal’s VAR arguments. Instead of writing

flag fGetChar(char *pch);

⋮

if (fGetChar(&ch))

ch has the new character...

you could write:

flag fGetChar(char &ch);

⋮

if (fGetChar(ch))    /*  &ch is actually passed  */

ch has the new character...

On the surface, this appears like a good addition since programmers can’t “forget” the explicit & required in regular C. But why would using this feature result in bug-prone rather than bug-resistant interfaces?

6.

The standard strcmp function takes two strings and compares them, character by character. If the two strings are equal, strcmp returns 0, if the first is less than the second, it returns a negative number, and if the first is greater than the second, it returns a positive number. So, when you call strcmp, the code usually looks like:

if (strcmp(str1, str2) rel_op Ø)

⋮

where rel_op is one of ==, !=, > , >=, <, or <=. This works, but the code is meaningless unless you’re familiar with the strcmp function. Describe at least two other function interfaces for string comparisons. The interfaces should be both more bug-resistant and more readable than strcmp’s interface.

PROJECT: Review the standard functions in whatever programming language you normally use and redesign the interfaces so that they will be more bug-resistant. What are the pros and cons of renaming the functions so they are more intelligible?

6

RISKY BUSINESS

If you were to put a programmer at the top of a cliff and give him a rope and a hang glider, how do you think he’d get to the bottom? Would he climb down the rope, or would he glide to the bottom? I have no idea whether he’d use the rope or the hang glider, but I’ll bet you anything that he wouldn’t jump to the bottom—it’s too risky. But for some reason, when programmers can choose among several possible implementations, they often consider only size and speed and completely ignore risk. What if the programmer on that cliff ignored risk and instead just took the most efficient route to the bottom? Geronimoooooooooo. . .

There are at least two reasons why programmers ignore risk.

Programmers ignore risk, in part, because they blindly assume that no matter how they implement their code, they’re going to implement it without bugs. Nobody says, “Guess what, I’m going to write a quicksort routine and I plan to have three bugs in it.” Programmers don’t plan to have bugs; they’re just not very surprised when bugs show up later.

But the major reason, I believe, that programmers ignore risk is that they have never been taught to ask questions such as, “How risky is this design?” “How risky is this implementation?” “Is there a safer way to write this expression?” “Is it even possible to test this design?” To ask questions like these, you must let go of the belief that regardless of your choices, you’ll end up with bug-free code. That might even be a well-founded belief, but the question in that case is when will you have bug-free code? Will it be hours or days from now because you use safe coding practices, or will it be weeks or months from now because you ignore risk and have lots of bugs to track down and fix?

Within any programming language there are risky, bug-prone coding practices. Let’s take a look at some examples from programmers using the C programming language.

DOTH YOUR DATA RUN OVER?

Some of the most sinister bugs are those in which the code appears obviously correct yet fails because of a subtle implementation problem. The code below, which initializes a lookup table for C’s standard tolower macro, suffers from this:

#include <limits.h>        /*  Pull in UCHAR_MAX  */

⋮

char chToLower[UCHAR_MAX+1];

void BuildToLowerTable(void)     /*  ASCII version  */

{

unsigned char ch;

/*  First set every character to itself  */

for (ch = Ø; ch <= UCHAR_MAX; ch++)

chToLower[ch] = ch;

/*  Poke lowercase letters into uppercase slots  */

for (ch = 'A'; ch <= 'Z'; ch++)

chToLower[ch] = ch + 32;

}

⋮

#define tolower(ch) (chToLower[(unsigned char)(ch)])

Despite how solid the code looks, BuildToLowerTable will probably hang your application, and possibly your system. Take a look at the test in the first loop. When will ch be greater than UCHAR_MAX? If you guessed Never, you’d be right. If you didn’t guess that way, let me explain.

Suppose ch is equal to UCHAR_MAX and the loop executes for what you expect to be the last time. Then, just before the final test, ch is incremented to UCHAR_MAX+1, causing it to overflow and wrap to 0. The code hangs in an infinite loop because ch will always be less than or equal to UCHAR_MAX.

How obvious is that problem when you look at the code?

You can also underflow a variable and find yourself in a similar predicament. Below is an implementation for C’s memchr function, which searches a block of memory for the first occurrence of a character. If it finds the character in the block, it returns a pointer to the character’s location; otherwise, it returns the NULL pointer. And like BuildToLowerTable that we just saw, the code for memchr seems correct when you read it but fails nevertheless.

void *memchr(void *pv, unsigned char ch, size_t size)

{

unsigned char *pch = (unsigned char *)pv;

while (--size >= Ø)

{

if (*pch == ch)

return (pch);        /*  Return  */

pch++;

}

return (NULL);

}

When will the loop terminate? When size is less than 0, of course, but will that ever be true? No, because size is an unsigned value—when it reaches 0, the expression ‑‑size will cause it to underflow and wrap to the largest unsigned value defined by the type size_t.

This underflow bug is worse than the bug in BuildToLowerTable because memchr will work correctly as long as it finds ch in the memory run. Even if it doesn’t find the character, it probably won’t hang your application or your system—it’ll just keep searching memory until it finds ch somewhere, and return a pointer to that character. That could be a hard bug to spot.

If you step through your code as I suggested in Chapter 4, you would pounce on both bugs. You would see that ch overflows and wraps to 0, and you would see that size underflows to the largest size_t value. You could peruse your source code for hours and never spot overflow and underflow bugs because they are so subtle, but if you look at the data flow in a debugger, such bugs become obvious.

Always ask, “Can this variable or expression overflow or underflow?”

CLOSE COUNTS ONLY IN HORSESHOES

You can see another often-used overflow example in the code below, which converts an integer to its ASCII representation:

void IntToStr(int i, char *str)

{

char *strDigits;

if (i < Ø)

{

*str++ = '-';

i = -i;          /*  Strip i's negative sign  */

}

/*  Derive the digits in reverse order  */

strDigits = str;

do

*str++ = (i % 1Ø) + 'Ø';

while ((i /= 1Ø) > Ø);

*str = '\Ø';

ReverseStr(strDigits);    /*  Unreverse the digits  */

}

This code breaks on two’s-complement machines for the single case when i is equal to the smallest negative number (e.g., -32768 on a 16-bit machine). The reason usually given is that the ‑i in the expression i = ‑i; overflows the range of the int type, and it’s perfectly true, but the real bug lies in the way the programmer implemented the code: he didn’t implement his design, he implemented something almost the same.

The design says, “if i is negative, stuff in a minus sign, and convert i’s unsigned counterpart to ASCII.” But that’s not what the code does. It actually implements “if i is negative, stuff in a minus sign, and convert i’s positive but signed counterpart to ASCII.” It’s the signed math that causes all the trouble. If you follow the algorithm and use unsigned math, the code works fine, plus you can break the code into two, more useful, functions:

void UnsToStr(unsigned u, char *str)

{

char *strStart = str;

do

*str++ = (u % 1Ø) + 'Ø';

while ((u /= 1Ø) > Ø);

*str = '\Ø';

ReverseStr(strStart);

}

void IntToStr(int i, char *str)

{

if (i < Ø)

{

*str++ = '-';

i = -i;

}

UnsToStr((unsigned)i, str);

}

You might be wondering why this works, given that it negates i just as in the previous example, it works because if i is the smallest negative number—0x8000 for a 16-bit int—and you negate it by “flipping all the bits and adding 1” you still get 0x8000 which looks like -32768 as a signed number, but as 32768 as an unsigned number. It’s all in how you interpret the bits. By definition, flipping the bits and adding one must give you the negative value of any two’s-complement number, but it’s up to you to correctly interpret the bit-pattern, and in this case, interpreting it as a signed value is wrong.

Still, being right is not always being smart. The code above feels wrong. It also assumes that -32768 is a valid int, which it isn’t, at least not if you’re sticking to portable types that state 16-bit ints have a valid range of -32,767 to 32,767. If you agree that -32768 is a non-portable int, then you can cast aside the whole mess with one well-placed assertion in IntToStr:

void IntToStr(int i, char *str)

{

/*  i out of range? Use LongToStr...  */

ASSERT(i >= -32767  &&  i <= 32767);

By using such assertions you not only avoid oddball problems related to one particular numbering system, but you also nudge other programmers into writing more portable code.

Implement your designs as accurately as possible. Being kinda close is being kinda buggy.

FUNCTIONS JUST “DOING THEIR THING”

I once did a thorough review of the code for Character Windows—a Windows-like library designed for Microsoft’s then character-based DOS applications in the mid-1980s. This was in the days before many desktop computers had the graphical user interfaces that we take for granted today. Back then, displays showed a fixed column of characters with a fixed number of rows of lines. No graphics anywhere. If your operating system has a “terminal” program, you know exactly what I’m talking about.

The two primary groups using the library, the Word and Works groups, felt that the code was bulky, sluggish, and unstable. I had just begun reviewing the code when I ran across an example of programmers not quite implementing what they had designed—and of the violation of another guiding principle for writing bug-free code.

But first some background.

The basic design for Character Windows was simple: The user viewed the video display as a set of windows, each of which could have its own subwindows. In the design, a root window represented the entire display, and this window had subwindows: a menu bar across the top of the screen, pull-down menus from that menu bar, application document windows, dialog boxes, and so on. Each of the windows in Character Windows could have its own subwindows. A dialog box, for example, could have subwindows for OK and Cancel buttons. The dialog box could have a listbox window that had its own subwindows for the contents of the list and for scrollbars. You get the idea.

To represent the hierarchical window structure, Character Windows used a binary tree in which one branch pointed to subwindows, called “children,” and the other branch pointed to windows with the same parent, called “siblings”:

typedef struct WINDOW

{

struct WINDOW *pwndChild;      /*  NULL if no children  */

struct WINDOW *pwndSibling;    /*  NULL if no siblings  */

char *strWndTitle;

⋮

} window;                          /*  Naming: wnd, *pwnd  */

You can turn to any algorithm book and find efficient routines to manipulate binary trees, so I was somewhat shocked when I reviewed the Character Windows code for inserting a child window into the tree. Despite the fact that the window structure was designed, on paper, to be a binary tree, it wasn’t implemented that way. Take a look at the code:

/*  pwndRootChildren is the pointer to the list of

*  top-level windows, such as the menu bar and the

*  main document windows.

*/

static window *pwndRootChildren = NULL;

void AddChild(window *pwndParent, window *pwndNewBorn)

{

/*  New windows cannot have siblings  */

ASSERT(pwndNewBorn->pwndSibling == NULL);

if (pwndParent == NULL)

{

/*  Add window to the top-level root list  */

pwndNewBorn->pwndSibling = pwndRootChildren;

pwndRootChildren = pwndNewBorn;

}

else

{

/*  If it’s the Parent's first child, start a new

*  sibling chain, otherwise add the child to the

*  end of the existing sibling chain.

*/

if (pwndParent->pwndChild == NULL)

pwndParent->pwndChild = pwndNewBorn;

else

{

window *pwnd = pwndParent->pwndChild;

while (pwnd->pwndSibling != NULL)

pwnd = pwnd->pwndSibling;

pwnd->pwndSibling = pwndNewBorn;

}

}

}

Since the root window (the one representing the entire display) never had siblings and never had a title and since you couldn’t move, hide, or delete it, the only field in the window structure that ever had any meaning was pwndChild—it pointed to the menu bar and application subwindows. That led somebody to decide that declaring an entire window structure for the root window was wasteful and replaced the wndRoot structure with pwndRootChildren, a simple pointer to the top-level windows.

Replacing wndRoot with a pointer may have saved a few bytes of data space, but the cost in code space was enormous. Instead of working with a simple binary tree, routines such as AddChild had to handle two different data structures: a linked list of window trees at the root level, and the window trees themselves. Worse, every routine that took a window pointer as an argument—and there were many—had to check for the special NULL pointer that represented the display “window.” No wonder the Word and Works groups were concerned about code bloat.

I didn’t bring up the problems with AddChild so that I could talk about design issues, but rather to point out that its implementation violated at least three guiding principles for writing bug-free code. You’ve seen two of these principles already:Don’t accept special purpose arguments such as the NULL pointer, and Implement your design, not something that approximates it. The third principle is new: Strive to make every function perform its task exactly one time .

What do I mean by that?

If you think about it, AddChild has one task, to add a child to an existing window, but if you look at the code, it has three separate insertion routines. Common sense tells you that if you have three pieces of code instead of one, you’re three times more likely to have bugs.

If you find yourself writing a function in which you do “the task” more than once, stop and ask yourself whether you can do the same job with one piece of code.

Sometimes you may want to write a function so that it does whatever it does more than once. The fast version of memset in Chapter 2 is an example of that—recall that it has two separate fill loops, a fast one and a slow one. You can break the guidelines, just be sure you’re doing it consciously, with good reason.

The first step to improving AddChild is easy enough: Rip out the “optimization” and implement the original design—a pure binary tree. To do that, you would replace pwndRootChildren with pwndDisplay, a pointer to a window structure representing the entire display. pwndDisplay would get allocated during program initialization calling the same pwndNewWindow function that would be called to create any new subwindow. In other words, the window representing the entire display is just another plain old boring window. Nothing special.

So now, instead of passing the special-case NULL to AddChild that represented the entire display, programmers would simply pass pwndDisplay, eliminating the need for any special-case code to handle root windows:

/*  The root level window will be allocated during program

*  initialization. pwndDisplay will be set to point to it.

*/

window *pwndDisplay = NULL;

void AddChild(window *pwndParent, window *pwndNewBorn)

{

/*  New windows cannot have siblings  */

ASSERT(pwndNewBorn->pwndSibling == NULL);

/*  If Parent's first child, start a new sibling

*  chain; otherwise add child to the end of the

*  existing sibling chain.

*/

if (pwndParent->pwndChild == NULL)

pwndParent->pwndChild = pwndNewBorn;

else

{

window *pwnd = pwndParent->pwndChild;

while (pwnd->pwndSibling != NULL)

pwnd = pwnd->pwndSibling;

pwnd->pwndSibling = pwndNewBorn;

}

}

The code above not only improves AddChild (and every other function that had to accommodate the oddball tree structure), but it fixes a bug in the original version whereby root windows were inserted backwards. Interestingly enough, that bug had been “fixed” in the Character Windows library by handling root-levels windows in reverse order everywhere it mattered—adding to the code bloat.

Implement “the task” just once.

NO ifs, &&S, OR BUTS

That last version of AddChild is much better than its predecessor, but it’s still doing twice the work it needs to do. The trip wire that should set off alarms in your head is the if statement, a sure sign that you’re probably doing the same work twice, although in different ways. True, there are cases in which you legitimately need an if statement in order to take some conditional action, but many times an if statement is the result of a sloppy implementation or design—it’s a lot easier to whip together a design filled with exceptions than it is to stop and derive a model without them.

For example, to traverse the sibling chain, you work with window structures and the pointers to the “next window,” but you can traverse the sibling chain in two ways: You can enter the loop pointing to the window structure and step from window to window as you loop, or you can enter the loop pointing to the “next window” pointer and step from pointer to pointer. You either use a window-centric algorithm, or a pointer-centric one. The current implementation for AddChild uses a window-centric algorithm.

But if you use the pointer-centric model, you’re always pointing at the “next window” pointer, and it doesn’t matter whether that “next window” pointer is the parent’s child pointer or merely a sibling pointer. This lets you eliminate the if statement required in the window-centric algorithm because there is no special case. It may be easier to understand this point if you compare the code below with the previous implementation:

void AddChild(window *pwndParent, window *pwndNewBorn)

{

window **ppwndNext;

/*  New windows cannot have siblings  */

ASSERT(pwndNewBorn->pwndSibling == NULL);

/*  Traverse the sibling chain using a pointer-

*  centric algorithm. We set ppwndNext to point at

*  pwndParent->pwndChild since the latter pointer

*  is the first "next sibling pointer" of the list.

*/

ppwndNext = &pwndParent->pwndChild;

while (*ppwndNext != NULL)

ppwndNext = &(*ppwndNext)->pwndSibling;

*ppwndNext = pwndNewBorn;

}

Don’t be surprised if the code above seems familiar. It should. After all, it’s a minor variant of the classic “dummy header” linked-list insertion algorithm famous for handling empty lists without any special-case code.

If you’re concerned that this version of AddChild violates my earlier advice about implementing your design and not something that approximates it, you needn’t be. The code may not implement the design the way you normally think about it, but it does implement it faithfully. It’s like looking at a lens in a pair of glasses—is the lens concave or convex? It could be either, depending upon how you view it. For AddChild, using a pointer-centric algorithm lets you write code without special cases.

If you’re worried about code efficiency, think about this: This final version of AddChild will generate much less code than any of the previous versions. Even the code for the loop will be comparable to—and possibly better than—the code generated for previous versions. Don’t let those extra *s and &s trick you into thinking the loop is doing more than before—it’s not. Compile the two and see for yourself.

Get rid of extraneous if statements

THE ?: OPERATOR IS AN if STATEMENT TOO

C programmers must regularly forget that the ?: operator is nothing but an if-else statement in disguise; nothing else adequately explains why programmers write code using ?: that they would never write using explicit if-else statements.

I ran across a good example of this in Excel’s dialog handling code. The code contained the function below, which determines the “next state” for a checkbox:

/*  uCycleCheckBox -- return the next state for a checkbox

*

*  Given the current setting, uCur, return what the next

*  checkbox state should be. This function handles both

*  two-state checkboxes that toggle between Ø and 1, and

*  three-state checkboxes that cycle through 2, 3, 4, 2...

*/

unsigned uCycleCheckBox(unsigned uCur)

{

return ((uCur<=1) ? (uCur?Ø:1) : (uCur==4)?2:(uCur+1));

}

I’ve worked with programmers who wouldn’t think twice about writing uCycleCheckBox using the nested ?: seen here, but these same programmers would switch to coding in COBOL before putting their name on the version below that uses explicit ifs:

unsigned uCycleCheckBox(unsigned uCur)

{

unsigned uRet;

if (uCur <= 1)

{

if (uCur != Ø)   /* Handle the Ø, 1, Ø... cycle */

uRet = Ø;

else

uRet = 1;

}

else

{

if (uCur == 4)   /* Handle the 2, 3, 4, 2... cycle */

uRet = 2;

else

uRet = uCur+1;

}

return (uRet);

}

Any modern-day compiler would generate nearly identical code for both the ?: version of uCycleCheckBox and the explicit if-else version.

The problem with the ?: operator is that, because it’s concise and easy to use; it appears ideal for producing efficient code. Programmers don’t even look for better solutions. Worse, programmers will collapse the if version into the ?: version to get a “better” solution that isn’t better at all.

It makes about as much sense as exchanging a $100 bill for 10,000 pennies so that you’ll have more money.

If these programmers took the time to derive an alternative algorithm instead of expressing the same algorithm in a slightly different way, they might come up with the following straightforward implementation:

unsigned uCycleCheckBox(unsigned uCur)

{

ASSERT(uCur >= Ø  &&  uCur <= 4);

if (uCur == 1)     /*  Time to restart the first cycle?  */

return (Ø);

if (uCur >= 4)     /*  What about the second one?  */

return (2);

return (uCur+1);   /*  Nope, nothing special this time.  */

}

Notice how the assertion validates uCur, but the test for the second cycle employs defensive programming in case uCur is invalid. This is an example of using defensive programming to protect your end-users in ship code, coupled with an assertion in your DEBUG code to automatically alert you to possible bugs.

Let me bring up another point. Some programmers are dogmatic that no function should ever have more than one return point. Look at this last implementation of uCycleCheckBox—it has not one, but three return points. Horrors!

What’s important to understand about such dogmatic guidelines is that they are guidelines, not rules. Guidelines are created to steer you clear of problems, and the long-lived guideline about having multiple returns in a function is that it can make a function unclear and difficult to maintain.

I agree with that guideline, as a guideline. Not as a rule.

Look at that last version of uCycleCheckBox. Is that function unclear in any way? Of course not. That code is as clear as water and is a much better solution than the two previous versions that studiously avoid using multiple returns.

Now let’s take a look at another possible solution for uCycleCheckBox, a table-driven solution:

unsigned uCycleCheckBox(unsigned uCur)

{

static const unsigned uNextState[] = { 1, Ø, 3, 4, 2 };

ASSERT(uCur >= Ø  &&  uCur <= 4);

return (uNextState[uCur]);

}

Compare this table implementation with any of the previous implementations. Which is the easiest to understand? Which generates the best code? And which is most likely to be correct the very first time? That should tell you something.

No, this version doesn’t incorporate any defensive programming in case there’s a bug and uCur is out-of-range, but the ASSERT would alert you to the problem in the DEBUG version of your code. You could, of course, add defensive code to this table-driven solution rather easily.

Avoid using nested ?: operators.

RID YOUR CODE OF REDUNDANCY

Here’s an obvious point: If you find that you must support a special case, at least try to isolate the code so that the details aren’t sprinkled throughout the function where a maintenance programmer might later miss them and unwittingly introduce bugs.

Earlier I showed you two implementations for IntToStr. What I didn’t show you is the way IntToStr is often shown in C programming books—although there it’s called itoa. The code usually looks something like this:

void IntToStr(int i, char *str)

{

int  iOriginal = i;

char *pch;

if (iOriginal < Ø)

i = -i;              /*  Strip i's negative sign  */

/*  Derive the string in reverse order  */

pch = str;

do

*pch++ = (i % 1Ø) + 'Ø';

while ((i /= 1Ø) > Ø);

if (iOriginal < Ø)       /*  Don't forget the '-'  */

*pch++ = '-';

*pch = '\Ø';

ReverseStr(str);         /*  Unreverse the string  */

}

Notice that the two if statements in the code are testing for the same special case. My question is simply why, when, as we saw in earlier examples, it’s so easy to wrap both bodies of code under a single if statement.

Sometimes repeated tests aren’t in if statements but in the conditions of for or while statements. For instance, take a look at yet another possible implementation for C’s memchr function:

void *memchr(void *pv, unsigned char ch, size_t size)

{

unsigned char *pch = (unsigned char *)pv;

unsigned char *pchEnd = pch + size;

while (pch < pchEnd  &&  *pch != ch)

pch++;

return ((pch < pchEnd) ? pch : NULL);

}

But compare that version to this one:

void *memchr(void *pv, unsigned char ch, size_t size)

{

unsigned char *pch = (unsigned char *)pv;

unsigned char *pchEnd = pch + size;

while (pch < pchEnd)

{

if (*pch == ch)

return (pch);    /*  Return  */

pch++;

}

return (NULL);

}

Which looks better to you, the first one that compares pch to pchEnd twice, or the second one that compares pch to pchEnd only once? Which is easier to figure out? And the crucial question: Which is more likely to be correct the first time you execute the code?

By localizing the block range check in the while condition, the second version is easier to understand and does exactly what it needs to do, and no more.

Handle your special cases just once.

HIGH RISK, NO RETURN

If those last two versions of memchr look correct to you, look again—they share the same subtle bug. Do you see it? Here’s a hint: What range of memory would memchr search when pv points to the last 72 bytes of physical memory space and size is also 72? If you said “it wouldn’t search any memory,” you’d be right. Those versions of memchr can cause pchEnd to overflow, wrapping it to address 0. The code uses a risky language idiom—and Risk wins.

A risky language idiom is any phrase or expression that appears to work correctly but in fact fails for some specific cases. The C language is loaded with such phrases, and you need to avoid them whenever possible. Here’s the risky idiom in memchr:

pchEnd = pch + size;

while (pch < pchEnd)

⋮

Here, pchEnd is set to point to the memory location right after the last character to be searched so that it can be used in the while expression. While this may be convenient for programmers, it works only if such a memory location exists, and if you’re searching right up to the end of memory, the location does not exist. The one exception to this—if you’re using ANSI C—is that you can always compute the address of the first element beyond the end of a named array. ANSI C requires implementations to support this capability.

As a first attempt to fix the bug, you might rewrite the code so that it tests against the last legal memory location:

pchEnd = pch + (size–1);

while (pch <= pchEnd)

⋮

but that doesn’t work either. Remember the UCHAR_MAX overflow bug we saw earlier in BuildToLowerTable? You have the same bug here. pchEnd may now point to a legal memory location, but the loop will never end because every time pch is bumped to pchEnd+1, it overflows.

The safe way to cover a range when you have both a pointer and a counter is to use the counter as the control expression:

void *memchr(void *pv, unsigned char ch, size_t size)

{

unsigned char *pch = (unsigned char *)pv;

while (size-- > Ø)

{

if (*pch == ch)

return (pch);     /*  Return  */

pch++;

}

return (NULL);

}

The code above is not only correct, but may also generate better code than the previous versions because it does not have to initialize pchEnd. A common belief is that the size-- version will be larger and slower than the pchEnd version because size must be duplicated for the imminent test against 0 before it can be decremented. The reality, though, is that the difference in size and speed is so inconsequential as to be unnoticeable.

This brings up another language idiom that I touched upon earlier. Some programmers would urge you to rewrite the loop expression using ‑‑size instead of size‑‑:

while (--size >= Ø)

⋮

The rationale for the change is that writing the expression above should never generate worse code than before but may in some cases generate slightly better code. The only problem with that advice is that if you blindly follow it, bugs will swoop down on your code like vultures to a carcass.

Why?

Well, for starters, the expression never works if size is an unsigned value (as it is in memchr) because unsigned values, by definition, will always be greater than or equal to 0. The loop will execute forever. Oops.

The expression doesn’t work properly for signed values either. What happens if size is an int and it enters the loop with the most negative value possible, INT_MIN? The answer is that size will be predecremented and will underflow, causing the loop to execute a large number of times instead of not at all. Oops again.

Using size-- > 0 works correctly no matter how you declare size—a subtle, but important distinction.

The only reason programmers use --size >= 0 is to gain some efficiency, but let’s take a look at that rationale for a moment. If you really have a speed problem, making such a minuscule improvement would be about as effective as cutting your lawn with nail clippers—you can do it, but one snip isn’t going to show. And if you don’t have a speed problem, why take the risk? Just as it’s not important for every blade to of grass to be exactly the same length, it’s not important that every line of code be optimally efficient. What is important is the overall effect.

For some programmers, the idea of tossing aside any possible efficiency gain seems almost criminal. But, as you’ve seen throughout this book, the idea is to systematically reduce risk by using safer designs and implementations even though they might be slightly less efficient. Users won’t notice if you slip in a few extra cycles here and there, but they will notice the occasional bug you introduce as you try to save those cycles. In investment terms, the return doesn’t justify the risk.

Another risky idiom that falls into the category of “wasted efficiency” is using bitwise operators to multiply, divide, and mod values by a power of 2. For example, the fast version of memset I showed you in Chapter 2 had these lines:

pb = (byte *)longfill((long *)pb, l, size / 4);

size = size % 4;

I’m sure some programmers read that code and thought, “How inefficient!” Those are the same programmers who would have written the division and the modulo operations using bitwise operators:

pb = (byte *)longfill((long *)pb, l, size >> 2);

size = size & 3;

It may be true that using the bitwise operators is faster than dividing or mod’ing with many processors, but it’s also true that dividing or mod’ing an unsigned value (such as size) by a perfect power of 2 is such a basic inefficiency—along with adding 0, and multiplying by 1—that any modern compiler would routinely optimize these expressions for you if it would be more efficient for your target machine. There is no reason to hand-optimize these unsigned expressions.

But what about signed expressions? Are the explicit optimizations worthwhile for those? Well, they are and they aren’t.

Suppose you have a signed expression such as this one:

midpoint = (upper + lower) / 2;

A two’s-complement compiler would not optimize the division to a shift because shifting a negative value would give you a different result than a signed division would. But if you knew that upper+lower always resulted in a positive value, you could rewrite the expression using a shift to get faster code:

midpoint = (upper + lower) >> 1;

So, yes, explicitly optimizing a signed expression can be worthwhile if it’s in a speed bottleneck. The question, though, is whether shifting is the best way to do it. And the answer is No. In C, casting works just as well and is far safer than shifting. Try this with your compiler:

midpoint = (unsigned)(upper + lower) / 2;

The idea is not to tell the compiler what to do, but rather to give it the information it needs to do the optimization for you. By telling the compiler that the sum of upper+lower is unsigned, you grant it permission to shift. Compare the cast with the shift. Which is easier to understand? Which is more portable? Which is more likely to be correct the first time?

Over the years, I have tracked down bugs in which programmers used shifts to divide signed values that weren’t guaranteed to be positive. I’ve tracked down bugs in which programmers shifted in the wrong direction. I’ve tracked down bugs in which programmers used the wrong shift count. I’ve even tracked down bugs in which programmers introduced precedence errors by carelessly converting expressions like a=b+c/4 to a=b+c>>2. I don’t ever recall tracking down a bug in which a programmer meant to divide by 4 and made a mistake typing the characters / and 4.

There are many risky language idioms in C as well as in the primary language you use to develop your own code. The best way for you to uncover those risky language idioms is to look at every bug you discover and ask yourself the question I’ve given you before: “How could I have prevented this bug?” You’ll soon develop your personal list of risky idioms to avoid.

Avoid risky language idioms.

INCONSISTENCY, THE GREMLIN OF CODE

Take a look at the code below, which contains one of the easiest kinds of bugs to keep out of your code: a precedence bug.

word = high<<8 + low;

The code is supposed to pack two 8-bit bytes into a 16-bit word, but because the + operator has a higher precedence than the shift operator, that’s not what it does—it shifts high by 8+low. The bug is understandable because C programmers don’t normally mix the bitwise and arithmetic operators. But why mix bitwise and arithmetic operators when it’s just as easy to stick with one type or the other?

word = high<<8  | low;    /*  bitwise solution  */

word = high*256 + low;    /*  arithmetic solution  */

Are these examples any harder to understand than the first? Are they any less efficient than the first? Of course not. But there’s one big difference: Both of these solutions are correct.

When programmers write expressions that contain just one kind of operator, they have a better chance of writing bug-free code because intuitively they know the precedence order within each group of operators. Sure, there are exceptions, but as a rule it’s true. How many programmers do you know who would write

midpoint = upper + lower / 2;

and expect the addition to happen before the division?

C Programmers don’t seem to have much trouble remembering the precedence order of the bitwise operators either—I suspect because memories of their Logic 101 courses linger on, where they played with functions like f(A,B ,C)=ĀB+C. Most programmers know that the order from high to low is ~, &, and then |, and it doesn’t take much extra thought to squeeze the shift operators between ~ and &.

Programmers tend to know the precedence order within groups of operator types, and it’s not until they start mixing operator types that they run into trouble. So the first guideline is: Don’t mix operator types if you don’t have to. The second guideline is: If you must mix operator types, use parentheses to isolate them .

You’ve already seen how the first guideline can protect you from bugs. You can see how the second guideline protects you by taking another look at the while loop we saw in the first exercise in Chapter 1:

while (ch=getchar() != EOF)

⋮

That loop mixes an assignment operator with a comparison operator and introduces a precedence bug. You could fix the bug by rewriting the loop without mixing operators, but the result looks terrible:

do

{

ch = getchar();

if (ch == EOF)

break;

⋮

} while (TRUE);

In this case, it’s better to ignore the first guideline and instead apply the second by separating the operations with parentheses:

while ((ch=getchar()) != EOF)

⋮

Don’t needlessly mix operator types.

If you must mix operators, use parentheses to isolate the operations.

DON’T ASSOCIATE WITH FAILURES

In Chapter 5, I pointed out that if your functions return errors disguised as regular values, it’s easier for programmers to mishandle or ignore those error conditions. I suggested that you simply design your functions so that they don’t return errors. In this chapter I’m going to turn that around and say, Don’t call functions that return errors. That way, you won’t mishandle or ignore an error condition returned by somebody else’s function. Sometimes you have no choice, and in those cases, be sure to walk through your error handling code in a debugger to be sure it works.

There is one bit of advice I want to stress: If you repeatedly handle the same error condition throughout your program, isolate that error handling . The simplest approach, the one every programmer already knows about, is to localize error handling in a subroutine. That works fine, but in some cases you can do even better.

Suppose that Character Windows had code to rename a window in half a dozen spots. The code below changes the window title if it can grab enough memory to hold the new title; otherwise, it keeps the current title and tries to handle the error condition somehow:

if (fResizeMemory(&pwnd->strWndTitle, strlen(strNewTitle)+1))

strcpy(pwnd->strWndTitle, strNewTitle);

else

/*  Unable to allocate space for the window title  */ ;

As I said, this code changes the window title if it can allocate enough memory to store the new title; otherwise it keeps the current title and tries to handle the error condition.

The question though, is how do you handle the error? Do you alert the user? Do you ignore the request and silently keep the old title? Do you copy a truncated version of the new title over top of the current one? Hmm. None of those solutions is ideal, particularly if the code is part of a general subroutine.

This is one of those cases in which you just don’t want the code to fail—ever. You always want to be able to rename a window. And you can.

The problem with the code above is that you’re not guaranteed to have enough memory for the new window title. But that’s easy to guarantee if you’re willing to over-allocate the title memory. For instance, in a typical Character Windows application, there were only a handful of windows that you would ever have renamed, and none of those windows’ titles took much memory, even at maximum length. Instead of allocating just the memory you needed for the title strings, you could always allocate enough memory to hold the largest possible title.

Renaming a window then becomes a simple string copy:

strcpy(pwnd->strWndTitle, strNewTitle);

Better still, you could hide the implementation in a RenameWindow function and use assertions to verify that the allocated title memory is large enough to hold any possible title:

void RenameWindow(window *pwnd, char *strNewTitle)

{

ASSERT(fValidWindow(pwnd));

ASSERT(strNewTitle != NULL);

ASSERT(fValidPointer(pwnd->strWndTitle, sizeMaxWndTitle));

strcpy(pwnd->strWndTitle, strNewTitle);

}

The obvious drawback to this approach is that you waste memory. But at the same time, you regain code space because you don’t need any error handling code. Your job is to weigh data-space against code-space and decide which is more important in each case you run across.

Avoid calling functions that return errors.

A SLAP ON THE RISK

By now, you should have a good idea of what I meant when I said that programming is “risky business.” All of the points in this chapter focus on trading a risky coding practice for one that produces results comparable in size and speed, and but is less error-prone.

But don’t stop with these points. Pull out your listings and take a hard look at the way you code. Did you think through all your coding habits, or did you adopt them because you saw other programmers using them?

Entry level programmers often think that shifting to divide is a “trick,” and experienced programmers think it’s perfectly obvious and have no qualms about doing it. But should they? Who is really right here?

QUICK REVIEW

◆

Remember that it’s possible for your algorithm to be correct but still have bugs because of less-than-ideal characteristics of the hardware it runs on. In particular, always check that your calculations and tests don’t overflow or underflow your data types.

◆

Be faithful to your design. The easiest way to introduce subtle bugs is to cheat on the implementation.

◆

Every function should have one well-defined task, but more than that, it should have only one way to accomplish that task. If the same code executes regardless of the inputs, you decrease the odds of having undetected bugs.

◆

if statements, in particular, are good warning signs that you may be doing more work than necessary. Strive to eliminate every unnecessary if statement in your code by asking yourself, “How can I change my design to remove this special case?” Sometimes you may have to alter your data structures, and at others you may have to alter the way you view them. Remember, is the lens concave or convex?

◆

Remember that if statements are sometimes disguised as control expressions in while and for loops. The ?: operator is another type of if statement.

◆

Be wary of risky language idioms—always keep an eye out for comparable but safer idioms. Pay particular attention to coding tweaks that supposedly give you better code. Since it’s rare for an implementation tweak to have any noticeable effect on overall efficiency, the extra risk is rarely worth the tradeoff.

◆

When you write expressions, try not to mix different types of operators. If you must mix operators, use parentheses to separate the operations.

◆

The special case of special cases is error handling. If possible, avoid calling functions that can fail. But if you must call a function that can return an error, try to localize the error—this will increase your chances of finding bugs in the error handling code.

◆

In some cases, it’s possible to eliminate general error handling by guaranteeing that what you want to do can’t fail. That may mean handling the error once during initialization or it may mean changing your design.

THINGS TO THINK ABOUT

1.

At one point I changed AddChild to use pwndDisplay instead of pwndRootChildren. Instead of using pwndDisplay, which points to an allocated window structure, I could have declared a global window structure, wndDisplay. Although that would have worked, why do you think I didn’t take that approach?

2.

Occasionally a programmer will ask whether, for efficiency, he or she should take a loop like

while (expression)

{

A;

if (f)    /*  f never changes throughout the loop. */

B;

else

C;

D;

}

and rewrite it as

if (f)

while (expression)

{

A;

B;

D;

}

else

while (expression)

{

A;

C;

D;

}

where A and D represent collections of statements. The second version will be faster, but how risky is it compared to the first version?

3.

Programmers regularly write functions that have several nearly identically-named arguments. For example:

int strcmp(const char *s1, const char *s2);

Why is using such similar names risky? How could you eliminate the risk?

4.

I’ve shown why it’s risky to use loop conditions such as:

while (pch++ <= pchEnd)

But why is it risky to use similar countdown loops?

while (pch-- >= pchStart)

5.

For efficiency or brevity, some C programmers take the shortcuts below. Why should you avoid them?

a.  Using printf(str); instead of printf("%s", str);.

b.  Using f = 1-f; instead of f = !f;.

c.  Using

int ch;    /*  ch *must* be an int. */

⋮

ch = *str++ = getchar();

⋮

PROJECT: Take a look at the programming language that you use and make a list of all the risky language features you can think of. For the C language, this would include falling through switch cases, arbitrary gotos, evaluating the same macro argument more than once, and so on. Once you have your list, write down the pros and cons of using each feature.

7

TREACHERIES OF

THE TRADE

When you write a mystery novel, you want every page to grip the reader. You want to evoke surprise, fear, and suspense in the reader. If you wrote, “Someone walked up and stabbed Joe,” you’d put your reader to sleep. For the reader to remain interested, you’d have to make her feel Joe’s fear with each footstep behind him. You’d have to make her experience Joe’s pounding heart as the steps got slowly closer. You’d have to raise in the reader a feeling of panic, like Joe’s, as the pace of the footsteps picked up. Most important, you’d have to keep the reader wondering: Will Joe get away?

Using surprise and suspense in a mystery novel is critical, but using them in code is terrible. When you write code, the “plot” should be so obvious and boring that other programmers know well in advance what’s going to happen. If your code has to have a someone walk up and stab Joe, then “So and So walked up and stabbed Joe” is exactly what you want. It’s short, it’s clear, and tells you everything you need to know.

But for some reason, programmers resist writing code that is obvious and boring. The urge to use tricks, to be clever, to do things out of the ordinary, seems to be overpowering.

In this chapter we’ll look at a few coding styles that don’t result in straightforward, boring code. These actual real-world examples are clever, tricky, and anything but obvious. And, of course, they all cause subtle bugs. Many of the examples in this chapter are an attempt to shave a byte of code or a few instruction cycles, and as I’ve pointed out a few times already, we’re not living in a world today where such optimizations matter anymore. Still the practices persist, so let’s talk about them…

THE NEED FOR SPEED

Here’s the bug-free version of memchr we saw in the last chapter.

void *memchr(void *pv, unsigned char ch, size_t size)

{

unsigned char *pch = (unsigned char *)pv;

while (size-- > Ø)

{

if (*pch == ch)

return (pch);

pch++;

}

return (NULL);

}

One of the games most programmers like to play is the “How can I make this code even faster?” game. That’s not a bad game to play, but, as we’ve seen throughout this book, it can have unexpected results if you take it to extremes.

If you played that game with the memchr code above, you’d ask yourself, “How can I speed up the loop?” There are only three possibilities: Remove the size check, remove the character test, or remove the pointer increment. It may seem impossible to rChapter 7emove any of those steps, but you can—if you’re willing to chuck traditional coding practices and try something daring.

Take a look at the size check. You need that check only so you can return NULL if you don’t find ch in the first size bytes of memory. To remove the check, you simply guarantee that you can always find ch. But how? Just temporarily store ch at the end of the memory run:

void *memchr(void *pv, unsigned char ch, size_t size)

{

unsigned char *pch = (unsigned char *)pv;

unsigned char *pchPlant;

unsigned char chSave;

/*  pchPlant points to the first character following

*  the memory run that memchr is searching. Plant

*  ch at that location so memchr is guaranteed to

*  find it even if it is not in the run.

*/

pchPlant = pch + size;

chSave = *pchPlant;    /*  Save original char  */

*pchPlant = ch;

while (*pch != ch)

pch++;

*pchPlant = chSave;    /*  Restore original char  */

return ((pch == pchPlant) ? NULL : pch);

}

Clever, right? By blotting out the character that pchPlant points to, you guarantee that memchr will find ch, and that allows you to remove the size check, doubling the speed of the loop.

But is it robust? Is it solid?

memchr may look robust in this new incarnation, particularly since it meticulously preserves the character that it changes, but this version of memchr has more problems than Batman has gadgets. For starters, think about these points:

◆

If pchPlant points to read-only-memory, storing ch at *pchPlant will have no effect and the function will return an invalid pointer if it can’t find ch in the first size+1 characters.

◆

If pchPlant points to memory-mapped I/O then storing ch at *pchPlant could cause weird interactions, anything from causing hard drives to go berserk or causing factory robots to go crazy with their welding torches.

◆

If pch points to the last size bytes of RAM, both pch and size will be legal, but pchPlant will point to nonexistent or write-protected memory. Storing ch at *pchPlant could cause a memory fault, or it could quietly do nothing and the function would fail if ch weren’t in the first size+1 characters.

◆

If pchPlant points to data that is shared by concurrent processes, one process storing ch at *pchPlant, could garble memory that another process might need to reference when it switches into context.

The last possibility is particularly troublesome because there are so many ways in which you can crash your system. What if you call memchr to search a block of memory that you’ve allocated and it garbles one of the memory manager’s data structures? If a concurrent process—a code thread or an interrupt routine, say—then switches into context, it had better not invoke the memory manager or the system may crash. What if you call memchr to scan a global array and it steps on an adjacent variable used by another task? Or what if two instances of your program try to search shared data in parallel? Any number of scenarios can kill your program.

Of course, you probably won’t realize that the optimized memchr causes subtle bugs, because, unless it modifies critical memory, it will appear to work fine. But when functions such as the optimized memchr do cause bugs, isolating those bugs is about as easy as finding your contact lens in the middle of a sandstorm. After all, the process executing memchr will work fine, while the other process—the one with mangled memory—crashes. You have no reason to suspect that memchr is the cause.

If you’ve ever wondered what those $100,000 in-circuit-emulators are for, now you know—they keep a record of every cycle, every instruction, and every piece of data the processor references up to the point of a crash. It may take you days to wade through the emulator’s output, but if you’re persistent and you don’t go blind staring at the reams of output, you should find the bug.

But why go through all that pain and effort? The alternative is so much easier: Don’t reference memory that you don’t own. And note, “Reference” means reading as well as writing. Reading unknown memory may not cause weird interactions with other processes, but it can kill your program just as quickly if you reference protected memory, non-existent memory, or memory mapped I/O.

Don’t reference memory that you don’t own.

A THIEF WITH A KEY IS STILL A THIEF

Curiously enough, I know programmers who would never reference memory they didn’t own but who would feel just fine writing code like the FreeWindowTree routine below:

void FreeWindowTree(window *pwndRoot)

{

if (pwndRoot != NULL)

{

window *pwnd;

/*  Release pwndRoot's children... */

pwnd = pwndRoot->pwndChild;

while (pwnd != NULL)

{

FreeWindowTree(pwnd);

pwnd = pwnd->pwndSibling;

}

if (pwndRoot->strWndTitle != NULL)

FreeMemory(pwndRoot->strWndTitle);

FreeMemory(pwndRoot);

}

}

Take a look at the while loop. Do you see the problem with it? As FreeWindowTree releases each child window in the linked list of siblings, it first frees pwnd, then references the freed block in the line:

pwnd = pwnd->pwndSibling

But what is the value of pwnd‑>pwndSibling once pwnd has been freed? It’s garbage, of course, but some programmers don’t accept that. The memory wasn’t garbage a nanosecond ago, and they haven’t done anything to affect it; therefore, they think, it should still be valid. They haven’t done anything, that is, but release it.

I’ve never understood why some programmers believe it’s permissible to reference memory they have released. How is that different from using a spare key to enter an apartment you once lived in, or to drive off in a car you once owned? You can’t safely reference freed memory because, as I pointed out in Chapter 3, the memory manager may use that storage for free-chains or for other private information.

Don’t reference memory that you have released.

TAKE ONLY WHAT YOU NEED

In the last chapter, I presented an implementation for the UnsToStr function. It looked like this:

/*  UnsToStr -- convert an unsigned value to a string. */

void UnsToStr(unsigned u, char *str)

{

char *strStart = str;

do

*str++ = (u % 1Ø) + 'Ø';

while ((u /= 1Ø) > Ø);

*str = '\Ø';

ReverseStr(strStart);

}

The code above is a straightforward implementation of UnsToStr , but no doubt there are programmers who feel uncomfortable with it because the code derives digits in reverse order, requiring a call to ReverseStr to reorder the digits. That seems wasteful. If you’re going to derive the digits in reverse order, why not build the string backwards and eliminate the need for ReverseStr? Why not, indeed:

void UnsToStr(unsigned u, char *str)

{

char *pch;

/*  u out of range? Use UlongToStr... */

ASSERT(u <= 65535);

/*  Store the digits in str from back to front. Start

*  storing the digits deep enough into the string to

*  hold the larget possible value for u.

*/

pch = &str[5];

*pch = '\Ø';

do

*--pch = (u % 1Ø) + 'Ø';

while ((u /= 1Ø) > Ø);

strcpy(str, pch);

}

Some programmers feel more comfortable with this code because it’s more efficient and easier to understand. UnsToStr is more efficient because strcpy (which you still need) is faster than ReverseStr, particularly for compilers that can generate the call as a few inline instructions. The code is easier to understand for C programmers because they’re familiar with the strcpy function. When C programmers see ReverseStr, they stumble for a moment much the way people do when they hear that their friend in the hospital is “ambulatory.”

So why am I talking about UnsToStr if it’s so perfect? Well, it’s not perfect. In fact, UnsToStr has a serious flaw.

Tell me, how much memory does the str parameter point to? You don’t know. But that’s not unusual for C interfaces. The unspoken rule between the caller and the implementor is that str will point to enough memory to hold the textual representation for u. But this optimized UnsToStr assumes that str points to enough memory to convert the largest possible value for u when that may not be the case. What if the caller wrote:

DisplayScore(unsigned UserScore)  /* UserScore is Ø to 25 */

{

char strScore[3];

UnsToStr(UserScore, strScore);

⋮

Since UserScore will never generate a string longer than 3 characters (two digits plus the nul character), it’s perfectly reasonable for a programmer to define strScore as a 3-character array. However, UnsToStr, as written in the previous example, assumes that the array strScore is large enough to hold the textual representation of the largest possible unsigned value, which could be many bytes longer than the 3 needed by DisplayScore. In the DisplayScore code, UnsToStr would typically—if you’re using a machine with a down-growing stack—destroy the frame back-pointer or the return address to DisplayScore’s caller or maybe both. You’d notice that problem since your application would likely crash. But if strScore weren’t the only local variable, you might not notice that UnsToStr was mangling the memory following strScore.

I’m sure there are programmers who will argue that it’s risky declaring strScore to be “just big enough” to hold the longest string that it needs. This is risky, but only because of programmers who write code like this last version of UnsToStr. It’s not necessary to be this tricky when you can implement UnsToStr efficiently and safely by building the string in a local buffer, and then copying the finished result to str:

void UnsToStr(unsigned u, char *str)

{

char strDigits[6];    /*  Conversion buffer  */

char *pch;

/*  u out of range? Use UlongToStr...  */

ASSERT(u <= 65535);

/*  Store the digits in strDigits from back to front.  */

pch = &strDigits[5];

*pch = '\Ø';

do

*--pch = (u % 1Ø) + 'Ø';

while ((u /= 1Ø) > Ø);

strcpy(str, pch);

}

You need to remember that, unless they are defined otherwise, pointers such as str don’t point to memory that you can use as workspace buffers. Pointers such as str are outputs that for efficiency’s sake are passed by reference instead of by value.

Don’t use output memory as workspace buffers.

KEEP PRIVATE THINGS TO YOURSELF

Of course, some programmers think that even calling strcpy in UnsToStr is too inefficient. After all, UnsToStr just created the output string. Why copy it to another buffer when you can save cycles by returning a pointer to the string you already have?

char *strFromUns(unsigned u)

{

static char *strDigits = "?????";   /* 5 chars + '\Ø' */

char *pch;

/*  u out of range? Use UlongToStr... */

ASSERT(u <= 65535);

/*  Store the digits in strDigits from back to front  */

pch = &strDigits[5];

ASSERT(*pch == '\Ø');

do

*--pch = (u % 1Ø) + 'Ø';

while ((u /= 1Ø) > Ø);

return (pch);

}

This code is nearly identical to the version of UnsToStr we saw in the last section, except that strDigits is declared to be static so that it will remain allocated even after strFromUns returns.

But imagine this: You have to implement a function in which you need to convert two unsigned values to strings so you write:

strHighScore = strFromUns(HighScore);

⋮

strThisScore = strFromUns(Score);

What’s wrong with that? Well, by calling strFromUns to convert Score, you destroy the string that strHighScore points to.

You could argue that the bug is in the code that calls strFromUns and not in strFromUns itself, but remember what we talked about in Chapter 5: It’s not enough that functions work correctly; they must also prevent programmers from making obvious mistakes. I would argue at the very least that strFromUns has an interface bug because you and I both know that programmers will make the mistake above.

Even if programmers are aware of the fragile existence of strFromUns’s strings, they can still introduce bugs without realizing they’re doing it. Suppose a programmer calls strFromUns and then calls another function, which unbeknownst to her, also calls strFromUns and destroys her string. Or suppose there are multiple code threads and one code thread calls strFromUns, wiping out a string still in use by another thread.

But even those points are minor compared to the bomb ticking in strFromUns, a bomb that will surely explode—probably many times—as your project evolves. Consider this: If you decide to insert a call to strFromUns into one of your functions:

◆

You must ensure that none of your callers (and callers to your callers, and so on) is still using a string that strFromUns returned to it. In other words, you must verify that no function in any of the possible call chains to your function assumes that strFromUns’s private buffer is preserved.

◆

You must also ensure that you don’t call any functions that call strFromUns, destroying a string that you still need. Of course, that means that you can’t call a function that calls a function (and so on) that calls strFromUns.

If you insert a call to strFromUns into one of your functions without performing these two checks, you risk introducing a bug. That’s bad enough. Imagine how much more difficult it is to adhere to those two conditions as programmers fix bugs and add new features. Every time they change a call chain to your function, or modify functions that your code calls, maintenance programmers must re-verify the two conditions. But do you think they will? Hardly. Those programmers won’t even realize that they should be verifying those conditions. After all, they’re just fixing bugs, rearranging code, and adding features. What does any of that have to do with strFromUns, a function they may never have used or even seen?

Functions such as strFromUns cause bugs again and again because their very design makes it easy to introduce bugs as programs are maintained. And, of course, when programmers isolate a strFromUns class of bug, the bug is not in strFromUns but in code that uses strFromUns incorrectly. Instead of fixing the true problem by rewriting strFromUns, programmers fix the specific bug and leave strFromUns in the program, ticking away…

Don’t pass data in static (or global) memory.

FUNCTIONAL LEECHES

Passing data in public buffers is risky, but you can get away with it if you’re careful and a bit lucky. But writing parasitic functions that rely on the internal workings of other functions is not only risky, but also irresponsible: If you change the host function, you kill the parasite.

The best example I know of a parasitic function is from, of all places, a widely ported, widely promoted standard implementation of the FORTH programming language. In the late 1970s and early 1980s, the FORTH Interest Group tried to stimulate interest in the FORTH language by providing public domain implementations of the FORTH-77 standard. Those FORTH implementations defined three standard functions: FILL, which filled a block of memory with a byte; CMOVE, which copied memory using a head-to-head algorithm; and <CMOVE, which copied memory using a tail-to-tail algorithm. CMOVE and <CMOVE were specifically defined as “head-to-head” and “tail-to-tail” moves so that programmers would know which function to use when they needed to copy overlapping memory blocks.

In the FORTH implementations, CMOVE was written in optimized assembly language, but for portability, FILL was written in FORTH itself. The code for CMOVE (translated into C here) was what you would expect:

/*  CMOVE -- move memory using a head-to-head move.  */

void CMOVE(byte *pbFrom, byte *pbTo, size_t size)

{

while (size-- > Ø)

*pbTo++ = *pbFrom++;

}

But the implementation for FILL was surprising:

/*  FILL -- fill a range of memory.  */

void FILL(byte *pb, size_t size, byte b)

{

if (size > Ø)

{

*pb = b;

CMOVE(pb, pb+1, size-1);

}

}

FILL calls CMOVE to do its job, which is surprising until you figure out how it works. Then the implementation is either “clever” or “gross,” depending upon your point of view. If you think that FILL is clever, consider this: the FORTH language may require that you implement CMOVE as a head-to-head move, but what if, for efficiency, you rewrite CMOVE so that it moves memory using longs instead of bytes? The answer, of course, is that you could write a blazing, bug-free version of CMOVE and break every function that calls FILL. To me, that’s not clever, that’s gross.

But let’s suppose that you know that CMOVE absolutely won’t change. You’ve even placed an ominous comment in CMOVE warning other programmers that FILL relies on its internal workings. That fixes only half the problems with CMOVE.

Suppose you’re working on the control code for a simple four-axis factory robot, where each axis has 256 positions. A simple design for such a robot would be to use 4 bytes of memory-mapped I/O such that each memory location would control a separate axis. To reposition an axis, you would write a value from 0 to 255 into its corresponding memory location. To retrieve the current position of an axis (which is especially useful while the axis is moving to a new position), you would read a byte from the corresponding memory location.

If you wanted to “home” all four axes to the (0,0,0,0) position, you could, in theory, write:

FILL(pbRobotDevice, 4, Ø);    /*  Put robot to bed.  */

Of course, that code doesn’t work given the way FILL is defined—FILL would write a 0 to the first axis and garbage to the other three axes, causing the robot to go wild. Why? If you look at FILL’s design, you can see that it fills memory by copying the previously stored byte to the current byte. But when FILL reads the first byte—expecting it to be 0—it reads instead the first axis’s current location, which is probably not 0 since the axis won’t have moved to position 0 in the nanosecond between storing the 0 and trying to read it back. That location could be any value, sending the second axis to some indeterminate spot. Of course, the third and fourth axes will be sent to similarly strange positions.

For FILL to work correctly, you would have to guarantee that it could read the same value from memory that it just wrote to memory. And you can’t guarantee that for memory mapped I/O.

But my point is that FILL is wrong because it peeks at the private details of another function and abuses that knowledge. That FILL doesn’t work correctly with anything but RAM is a secondary problem, but it again demonstrates that you ask for trouble any time you stray from straightforward, boring code.

Note that if CMOVE had included an assertion to verify that its arguments were valid (that is, that the source memory would not be destroyed before being copied to the destination), the programmer who wrote FILL would have gotten an assertion the first time the code was tested. That would have left the programmer with two choices: Rewrite FILL using a reasonable algorithm, or remove the assertion from CMOVE. Fortunately, very few programmers would remove such an assertion from CMOVE just so that FILL’s slimy implementation would work.

Don’t write parasitic functions.

THE OLD PAINT-STIRRER TRICK

One of the oldest tricks in home maintenance is to pick up a screwdriver to pry the lid off a can of paint and then use the screwdriver as a stir-stick. I should know; I have a collection of multicolored screwdrivers. Why do people use screwdrivers to stir paint when they know darn well they shouldn’t? I’ll tell you why: because at that moment the screwdriver is convenient, and it works. Likewise, there are programming tricks that are convenient and guaranteed to work and, like those screwdrivers, are not used for their intended purposes.

Take a look at the code below, which uses the result of a comparison as part of a computational expression.

unsigned atou(char *str);    /*  unsigned version of atoi  */

/*  atoi -- convert an ASCII string to an integer value  */

int atoi(char *str)

{

/*  str has the format "[white space][+/-]digits"  */

while (isspace(*str))

str++;

if (*str == '-')

return (-(int)atou(str + 1));   /*  RETURN  */

/*  Skip the optional '+' sign if there is one. */

return ((int)atou(str + (*str == '+')));

}

The code skips an optional leading + sign by adding the result of the test (*str == '+') to the string pointer. You can write such code because in C the result of any relational operation will be either 0 or 1. But what some programmers fail to recognize is that the ANSI C standard is not a rule book that tells you what you can and can’t do any more than the tax code tells you how to do your taxes. In both cases, you can adhere to the letter of the law but violate the intent.

The real problem in this example is not so much the code as in the programmer’s attitude. If a programmer feels comfortable using logical evaluations in computational expressions, what other shortcuts is he or she willing to take? How safe are those?

Don’t abuse your programming language.

APL SYNDROME

Programmers who aren’t aware of how source code translates into machine code will often try to improve the quality of the machine code by using tight language idioms. Their idea is that if you use a minimal amount of source code, you should get a minimal amount of machine code. And while there is a correlation between the size of your source code and the size of the corresponding machine code, that correlation breaks down when you apply it to small snippets of code.

Recall the uCycleCheckBox function from chapter 6:

unsigned uCycleCheckBox(unsigned uCur)

{

return ((uCur<=1) ? (uCur?Ø:1) : (uCur==4)?2:(uCur+1));

}

uCycleCheckBox may be tight C code, but as I’ve already pointed out, it generates terrible machine code. And what about the return statement we saw in the last section?

return ((int)atou(str + (*str == '+')));

Adding the result of a comparison to a pointer may generate decent code if you have a good optimizing compiler and your target machine can generate a 0/1 test result without using any branches. If that doesn’t describe your setup, more likely than not your compiler will internally expand the comparison to a ?: operation and generate machine code as though you had written this code:

return ((int)atou(str + ((*str == '+') ? 1 : Ø)));

Since a ?: operation is nothing but an if- else statement in disguise, you would get worse code than if you had written the obvious, boring, and straightforward version.

if (*str == '+')            /*  Skip the optional '+' */

str++;

return ((int)atou(str));

Of course, there are other ways to optimize the code. I’ve seen cases in which a programmer took a two-line if statement and “improved” it by replacing the if statement with an || operator:

(*str != '+')  ||  str++;   /*  Skip the optional '+' */

return ((int)atou(str));

Such code works in languages that use short-circuit evaluation for logical operators (as C-derived languages do), but fitting the code onto a single line doesn’t mean that you’ll get better machine code than if you’d used an if statement. In fact, you could generate worse code using || if your compiler unnecessarily generates a 0 or 1 result as a side effect.

A simple guideline is: Use || for logical expressions, use ?: for conditional expressions, and use if for conditional statements. Following this guideline may be downright boring, but your code will be more likely to be efficient and maintainable.

If you suffer from the dread “one-line-itis” disease (also known as “APL syndrome”) in which you constantly use bizarre expressions so that your code will fit on one line, get into your best yoga position, take a deep breath, and start repeating, “It is possible for efficient code to span multiple lines. It is possible for efficient code to span multiple lines...”

Concise source code does not guarantee efficient machine code.

NO HOITY-TOITY PROGRAMMING

Some computer experts can’t bring themselves to use plain, everyday English in their documentation and technical papers. Instead of saying, “That bug may hang or crash your system,” one of these experts would say “Such a software defect may cause a loss of system control or cause system termination.” These experts throw around terms like “axiomatic program verification” and “defect taxonomies” as though they were part of a programmer’s everyday vocabulary. Rather than helping readers, such experts confuse readers by burying their message in obscure terminology.

Writers aren’t alone in this tendency to sound impressive rather than simply being clear. Programmers do it too. For example, how does this function work?

void *memmove(void *pvTo, void *pvFrom, size_t size)

{

byte *pbTo   = (byte *)pvTo;

byte *pbFrom = (byte *)pvFrom;

((pbTo < pbFrom) ? tailmove : headmove)(pbTo, pbFrom, size);

return (pvTo);

}

Would you understand the function better if I rewrote it like this?

void *memmove(void *pvTo, void *pvFrom, size_t size)

{

byte *pbTo   = (byte *)pvTo;

byte *pbFrom = (byte *)pvFrom;

if (pbTo < pbFrom)

tailmove(pbTo, pbFrom, size);

else

headmove(pbTo, pbFrom, size);

return (pvTo);

}

The first example may not look like legal C, but it is. And odds are great that your compiler will generate smaller code for the first example than for the second for the simple reason that it will push all the arguments to the call stack before making the test whether to call tailmove or headmove. This is not a speed optimization for the compiler, but a size optimization.

Still, how many programmers will grasp how that first function works? What if they have to maintain that code? You’re not doing anybody any favors if you write correct code that nobody can understand. You might as well write it in hand-optimized assembly language.

Here’s another example that confuses many C programmers:

while (expression)

{

int i = 33;           /*  Declare locals.  */

char str[2Ø];

⋮

}

Quick! Is i initialized each time through the loop, or only the first time the loop is entered? Do you know the correct answer without thinking about it? If you’re not sure, you’re in good company—even expert C programmers usually pause for a few moments as they mentally scan C’s initializer rules.

What if I tweaked the code slightly?

while (expression)

{

int i;

char str[2Ø];

i = 33;

⋮

}

Do you have any doubt that i is set to 33 each time through the loop? Would any programmer on your team doubt it? Of course not.

Programmers frequently forget that they have two audiences: the customers who use their code, and the maintenance programmers who have to update that code. I don’t know many programmers who forget about their customers, but judging from the code I’ve read over the years, I’ve found that many programmers do tend to forget about their second audience, the maintenance programmers.

The idea that you should be writing maintainable code is nothing new. Programmers know they should be writing such code. What programmers don’t always realize, though, is that if they use language that only experts in their language can understand, their code is not maintainable. After all, maintainable code is, by definition, code that maintenance programmers can easily understand and modify without introducing bugs. And regardless of whether they should be, the maintenance programmers tend to be the newcomers to a project, not the experts who have been there awhile.

Good programmers write code that works. Great programmers write code that works and can be easily understood and maintained. Strive to be a great programmer.

Write straightforward, boring code that is easily understood.

WASTE THAT BAG OF TRICKS

We’ve looked at a number of questionable coding practices, many of which may look fine at first glance. But as we’ve also seen, the second, or even fifth glance may not alert you to the subtle side effects that tag along with “clever” code. If you find yourself writing code that feels tricky to you, stop and find another solution. If your code feels tricky, that’s your gut telling you that something isn’t right. Listen to your gut. If you find yourself thinking of a piece of your code as a neat trick, you’re really saying to yourself that an algorithm produces correct results even though it’s not apparent that it should. The bugs won’t be apparent to you either.

So be truly clever; write boring code. You’ll have fewer bugs and the maintenance programmers will love you for it.

QUICK REVIEW

◆

If you’re working with data that you don’t own, don’t write to it, even temporarily. And though you might think that reading from data is always safe, remember that reading from memory mapped I/O may be hazardous to your hardware.

◆

It may be tempting to reference memory once you have released it, but restrain yourself. There are too many ways in which referencing free memory can cause bugs.

◆

It may also be tempting, for efficiency, to pass data in global or static buffers, but that’s a shortcut fraught with dangers. If you write a function that creates data useful only to the caller, return the data to the caller, or guarantee that you won’t unexpectedly change that data.

◆

Don’t write functions that rely on the specific implementations of other functions. That FILL routine we saw had no business calling CMOVE the way it did. Such nonsense is suitable only as an example of bad programming.

◆

Avoid questionable programming idioms even though your programming language happens to guarantee (today) that they will work. Don’t live on the edge unless absolutely necessary.

◆

Logically, it would seem that the shorter your source code is, the more efficient the resultant machine code would be. It doesn’t work that way. Before you take a clean multiline piece of code and squash it into something that fits on one line, make sure that you’re getting better machine code for your trouble. Even then, remember that local efficiency gains are rarely noticeable and are not usually worth mucking up your code.

◆

Finally, don’t write code the way lawyers write contracts. If an average programmer can’t read and understand your code, then it’s too complicated; use simpler language.

THINGS TO THINK ABOUT

1.

C programmers regularly modify the arguments passed to functions. Why doesn’t this practice violate the write privilege for input data?

2.

I was once reading some C code in a journal when I noticed a function that set three local variables to 0 using, of all things, the memset function:

void DoSomething(...)

{

int i;

int j;

int k;

memset(&k, Ø, 3*sizeof(int));   /* Set i, j, k, to Ø */

⋮

Such code may work with some compilers, but why should you avoid using this trick?

3.

C has traditionally allowed programmers to pass fewer arguments to a function than the function expects to receive. Some programmers use that feature to optimize calls that don’t require all the arguments. For instance:

⋮

/*  No need to pass second arg for this op  */

DoOperation(opNegAcc);

⋮

void DoOperation(operation op, int val)

{

switch (op)

{

case opNegAcc:

accumulator = -accumulator;  /* val not needed */

break;

case opAddVal:

accumulator += val;

break;

⋮

Although the optimization works, why should you avoid it?

4.

In the C language, comparison operators like < and == are defined to evaluate to 1 or 0 for true and false. Knowing this, C programmers will sometimes write tests like the one below to verify that a boolean parameter passed to a function is 1 or 0:

ASSERT((f & 1) == f);

Using the bitwise “and” operator like that might correctly verify that f is 1 or 0, but why should the test be rewritten?

8

THE REST IS

ATTITUDE

Throughout this book, I’ve talked about techniques you can use to detect and to prevent bugs. Using these techniques won’t guarantee that you’ll write bug-free code any more than having a team of skillful ball players will guarantee that you have a winning team. The other necessary ingredient is a set of good habits, attitudes, and philosophies.

Would you expect those ball players to have a winning season if they grumbled all day about having to practice? What if they were constantly angry because their salary was a meager $3.2 million per year or were always worried about being traded or cut? These concerns have nothing to do with playing ball, but they have everything to do with how well the players perform.

You can use all of the suggestions in this book to help eliminate bugs, but if you have “buggy” philosophies or coding habits that cause bugs, you’re going to have a tough time writing bug-free code.

In this chapter, I’ll talk about some of the most common barriers to writing bug-free code. All are easily correctable; often all you need to do is become aware of them.

FOR MY NEXT TRICK, DISAPPEARING BUGS

How many times have you asked somebody about a bug they were fixing and heard in response, “Oh, that bug went away.”? I said that once, many years ago, to my very first manager. He asked me if I’d managed to track down a bug in the Apple II database product we were wrapping up, and I said, “Oh, that bug went away.” The manager paused for a moment, then asked me to follow him to his office, where we both sat down.

“Steve, what do you mean when you say, ‘the bug just went away’?”

“Well, you know, I went through the steps in the bug report, and the bug didn’t show up.”

My manager leaned back in his chair. “So what do you suppose happened to that bug?”

“I don’t know,” I said. “I guess it already got fixed.”

“But you don’t know that, do you?”

“Umm, no, I guess I don’t.” I admitted.

“Well don’t you think you had better find out what really happened? After all, you’re working with a computer; bugs don’t fix themselves.”

That manager then went on to explain the three reasons bugs disappear: the bug report was wrong, the bug has been fixed by another programmer, or the bug still exists but isn’t apparent. His final words on the subject were to remind me that, as a professional programmer, it was my job to determine which of the three cases described my disappearing bug and to act accordingly. In no case was I to simply ignore the bug because it had “disappeared.”

Lesson learned.

Programmers sometimes grumble when I ask them to drag out older sources to look for a reported bug; it seems like a waste of time. If it seems that way to you, consider that you’re not reverting to earlier sources on a whim. You’re looking at those sources because there is an excellent chance that there is a bug and looking at those older sources is the most efficient way to track it down.

Suppose you isolate the bug in those earlier sources and find that the bug has indeed been fixed in the current sources. Have you wasted your time? Hardly. After all, which is better, closing the bug as “fixed” or labeling it as “nonreproducible” and sending it back to the testing group? What will the testers do then? They certainly can’t assume that the bug has been fixed—their only two options are to spend additional time trying to reproduce the bug or to leave it marked as nonreproducible and hope that it was fixed. Both options are far worse than tracking down the bug in earlier sources and closing the bug as “fixed.”

Bugs don’t just “go away.”

A FIX IN TIME SAVES NINE

When I first joined the Microsoft Excel team, the practice was to postpone all bug-fixes to the end of the project. It’s not that the team had a cast-iron scroll staked to a wall that read, “Thou shalt not fix bugs until all features have been implemented,” but there was always pressure to keep to the schedule and knock out features. At the same time, there was very little pressure to fix bugs. I was once told, “Unless a bug crashes the system or holds up the testing group, don’t worry about fixing it. We’ll have plenty of time to fix bugs later, after we complete the scheduled features.” In short, fixing bugs was not a high priority.

I’m sure that sounds backwards to current Microsoft programmers because projects aren’t run that way anymore; there were too many problems with that approach, and the worst was that it was impossible to predict when you would finish the product. How do you estimate the time it takes to fix 1742 bugs? And of course, there aren’t just 1742 bugs to fix—programmers will introduce new bugs as they fix old ones. And (closely related) fixing one bug can expose other, latent, bugs that the testing group was unable to find because the first bug was getting in the way.

And those weren’t the only problems.

By finishing the features before fixing the bugs, the developers made the product look like it was much further along than it actually was. Key people in the company would use the internal releases, see that they worked except for the occasional bug, and wonder why it was taking Development six months to finish a nearly final product. They wouldn’t see out-of-memory bugs or the bugs in features they never tried. They just knew that the code was “feature complete” and that it basically appeared to work.

Fixing bugs for months on end didn’t do much for morale either. Programmers like to program, not to fix bugs, but at the end of every project they would spends months doing nothing but fixing bugs, often under much pressure because it was obvious to everybody outside Development that the product was nearly finished. Why couldn’t it be ready in time for the Consumer Electronics Show or Apple’s Developer Conference?

What a mess.

Then a run of buggy products, starting with Macintosh Excel 1.03 and ending with the cancellation—because of a runaway bug list—of an unannounced Windows product, forced Microsoft to take a hard look at the way it developed products. The findings were not too surprising:

◆

You don’t save time by fixing bugs late in the product cycle. In fact, you lose time because it’s often harder to fix bugs in code you wrote 6 months ago than in code you wrote days ago.

◆

Fixing bugs “as you go” provides damage control because the earlier you learn of your mistakes, the less likely you are to repeat those mistakes.

◆

Bugs are a form of negative feedback that keep fast but sloppy programmers in check. If you don’t allow programmers to work on new features until they have fixed all their bugs, then you prevent sloppy programmers from spreading half-implemented features throughout the product—they’re too busy fixing bugs. If you allow programmers to ignore their bugs, you lose that regulation.

◆

By keeping the bug count near zero, you have a much easier time predicting when you’ll finish the product. Instead of trying to guess how long it will take to finish 32 features and 1742 bugs, you just have to guess how long it will take to finish the 32 features. Even better, you’re often in a position to drop the unfinished features and ship what you have if you need a specific release date.

None of these points is uniquely suited to Microsoft development; they are general points that apply to any software development. If you are not already fixing bugs as you find them, let Microsoft’s negative experience be a lesson to you. You can learn through your own hard experience, or you can learn from the costly mistakes of others.

Don’t fix bugs later; fix them now.

BUG-DOCTOR TO THE RESCUE!

In his book Awaken the Giant Within, Anthony Robbins tells the story of a doctor who is standing beside a raging river when she hears the cry of a drowning man. The doctor looks around and seeing nobody else to help, jumps into the water. She swims out, brings the drowning man to shore, and gives him mouth-to-mouth resuscitation. She no sooner has the man breathing again when she hears two more cries from the river. She dives in and brings those two ashore. Just as she stabilizes those people, the doctor hears four cries for help. Then she hears eight cries for help… Unfortunately the doctor is so busy saving people that she has no time to go upstream to find out who’s throwing them in!

Like that doctor, programmers are sometimes so busy “healing” bugs that they never stop to figure out what’s causing them. The strFromUns function we talked about in Chapter 7 is an example of this problem. The strFromUns routine causes bugs because it forces programmers to pass data in static memory. But when bugs show up, they’re downstream in the callers to strFromUns, not in strFromUns itself. Which buggy routine do you think gets fixed, strFromUns—the true source of the bugs—or the functions that call strFromUns and wipe out the results of a previous call?

Another example of this problem occurred when I was porting a Windows Excel feature to Macintosh Excel. (They were still two independent bodies of source code at the time.) After I ported the feature, I began testing the code and found a function that was getting an unexpected NULL pointer. I looked at the code, but it wasn’t clear whether the bug was in the caller (passing NULL), or in the function (not handling NULL).

I went to the original programmer and explained the problem to him. He promptly loaded the source code file into an editor and said, “Oh, that function can’t take a NULL pointer.” Then, as I stood there watching, he fixed the bug by inserting a “quick escape” if the pointer was NULL :

if (pb == NULL)

return (FALSE);

I pointed out that if the function shouldn’t be getting a NULL pointer, then the bug is in the caller, not in the function, to which he replied, “I know the code; this will fix it.” And it did. But, to me the solution felt as if we’d fixed a symptom of the bug and not the cause of it, so I went back to to my office and spent 10 minutes tracking down the source of the NULL pointer. Not only was the NULL pointer the true bug, but it also accounted for two other known bugs.

Other times, I’ve tracked a bug to its source and then thought, “Wait, this can’t be right; if it is, this function over here would be broken too, and it’s not.” I’m sure you can guess why that other function worked. It worked because somebody used a local fix for a more general bug.

Fix the cause, not the symptom.

ARE YOU A CODE MEDDLER?

“If it ain’t broke, fix it anyway” seems to be the battle cry of some programmers. No matter how well a piece of code works, some programmers feel compelled to put their mark on it. If you’ve ever worked with a programmer who reformats entire files to suit his or her tastes, you know what I’m talking about. Most programmers are much more conservative than that about “cleaning up” code, but all programmers tend to clean up code to some degree.

The trouble with cleaning up code is that programmers don’t always treat their improved version of the code as if it were new code. There are programmers who, while scrolling through a file, might see the code below and feel compelled to change the test against 0 to a test against '\0 ‘; others might feel compelled to remove the test altogether.

char *strcpy(char *pchTo, char *pchFrom)

{

char *pchStart = pchTo;

while ((*pchTo++ = *pchFrom++) != Ø)

{}

return (pchStart);

}

The problem with changing the 0 to a nul character is that it’s easy to mistakenly type '0' instead of '\0', but how many programmers would bother to test strcpy after making such a simple change? Here’s a better question: When you make such simple changes, do you thoroughly test the code as though it were freshly written? If you don’t, you risk introducing bugs with those unnecessary changes.

You might think that some changes couldn’t possibly be wrong as long as the code still compiles. How could changing the name of a local variable, for example, cause problems? Well, it can. I once tracked a bug to a function which had a local variable named hPrint that was conflicting with a global variable of the same name. Since the function had worked until recently, I looked at the older sources to see what had changed and to verify that my fix would not reintroduce an earlier bug. What I found was a cleanup. The earlier version had a local variable named hPrint1, but no hPrint2 or hPrint3 to justify the ‘1’ in the name. Whoever removed the ‘1’ probably assumed that hPrint1 was an artifact from earlier days and cleaned it up, causing the name conflict and a bug.

To keep yourself from making the same kind of cleanup mistakes, tape yet another message to your monitor: The programmers I work with are not bozos.

That message should remind you that if you see code that is “obviously wrong,” or “clearly unnecessary,” proceed with utmost caution. Unless you’re willing to take the time to totally retest the code you change, resist the urge to “improve” the code. If the questionable code is that obvious, there is probably a good, but non-obvious reason why it’s there. I’ve seen ridiculous code whose only purpose was to work around an obscure compiler code-generation bug. Of course, such code should have a comment to explain what’s going on, but not all programmers are that thoughtful.

If you find code like

char chGetNext(void)

{

int ch;         /*  ch *must* be an int  */

ch = getchar();

return (chRemapChar(ch));

}

don’t clean up the function by removing the obviously “unnecessary” ch:

char chGetNext(void)

{

return (chRemapChar(getchar()));

}

If you remove the ch, you could introduce a bug if chRemapChar is a macro that evaluates its argument more than once. Keep the “unnecessary” local and prevent the unnecessary bug.

Don’t clean up code unless the cleanup is critical to your product’s success.

PUT “COOL” FEATURES INTO COLD STORAGE

Refraining from code cleanups is a specific case of a more general principle that results in fewer bugs: Don’t write (or change) code if you don’t have to. That may seem like strange advice, but you’d be surprised how often you can drop a feature by asking, “How important is this feature to the success of the product?”

Some features add no value to the product but exist merely to fill out feature sets; others exist because a large corporate customer asks for them; and still others exist because a competitor’s product has them and a reviewer somewhere decided to put them on a feature-list chart.

If you have a good marketing and product planning team, you shouldn’t run into any of these useless features. But as a programmer, you may run into or even originate unnecessary features.

Have you ever heard a programmer say something like, “It would be so cool if WordSmasher could do. . .”?

The question, though, is whether the feature is “cool” because it would improve the product or because implementing it would be technically challenging. If the feature will improve the product, postpone consideration of it until the next version of your program so that it can be properly evaluated and scheduled. If the feature is merely challenging, kill it. I don’t suggest this to stifle creativity; I suggest it to stifle needless feature growth and associated bugs.

Sometimes technically challenging features improve the product; sometimes they don’t. Choose carefully.

Don’t implement nonstrategic features.

NO FREE LUNCHES

“Free” features are another source of unnecessary bugs. On the surface, free features seem worthwhile because they fall out of existing designs with little or no effort. What could be better than that? But there’s one big fat problem with free features—they’re almost never critical to the success of the product. And of course, any noncritical feature is a potential source of bugs. Programmers add free features to a program because they can, not because they should. After all, why shouldn’t you add a feature if it doesn’t cost you anything?

There’s the fallacy. Free features may not cost much for the programmer, but there is more to a feature than code. Somebody has to write documentation for the feature. Somebody has to test the feature. And of course somebody has to fix any bugs that show up with the feature.

When I hear a programmer say that a feature is free, that tells me that he or she has not spent much time thinking about the true costs involved.

There are no free features.

FLEXIBILITY BREEDS BUGS

Another strategy you can use to prevent bugs is to strip unnecessary flexibility from your designs. You’ve seen me use this principle throughout this book. In Chapter 1, I used optional IDE warnings to disallow redundant and risky C language idioms. In Chapter 2, I defined ASSERT as a statement to prevent the macro from being mistakenly used in expressions. In Chapter 3, I used an assertion to catch NULL pointers passed to FreeMemory even though it’s quite legal to call C’s free function with a NULL pointer. From every chapter I could list examples in which I reduced flexibility in order to prevent bugs.

The trouble with flexible designs is that the more flexible they are, the harder it is to detect bugs. Do you remember the points I made about C’s realloc function in Chapter 5? You can throw almost any set of inputs at realloc and it will do something, even though that something may not be at all what you expect. Worse, it’s hard to detect realloc bugs because the function is so flexible that you can’t insert meaningful assertions to validate its inputs. But if you take a function like realloc and create separate functions for each of its distinct behaviors, you can validate those function’s arguments quite effectively.

In addition to unduly flexible functions, you should keep a wary eye open for unduly flexible features. Flexible features are troublesome because they can lead to unexpected “legal” situations that you didn’t think to test for or even realize that were legal.

When I was adding color support to Microsoft Excel for Apple’s then new Macintosh II machines, I ported code from Windows Excel that would allow users to specify the color of the text displayed in a spreadsheet cell. To add color to a cell, the user would take an existing cell format like the one below

$#,##Ø.ØØ      /*  Print 1234.5678 as $1,234.57. */

and tack a color specification onto the front of it. To display a number in blue, the user would change the format to:

[blue]$#,##Ø.ØØ

If the user typed [red], the number would be drawn in red, and so on.

Excel’s product specification was quite clear—the color specification should preface the number format—but after I ported the feature and began testing the code, I found that the formats below would work as well:

$#,##Ø.ØØ[blue]

$#,##[blue]Ø.ØØ

$[blue]#,##Ø.ØØ

The user could put the [blue] anywhere. When I asked the original programmer whether this was a bug or a feature, he said that the arbitrary placement of the color specification “just fell out of the parsing loop.” He didn’t see anything wrong with allowing a bit of extra flexibility—nor did I at the time—so the code remained that way. In retrospect, I see that we should never have allowed that extra flexibility.

It didn’t take the testing group long to find half a dozen subtle bugs, all ultimately traceable to the format parser, which did not expect to find color specifications in the middle of a format.

Unfortunately, instead of fixing the problem by removing the unnecessary flexibility—a fix that would have taken one simple if statement—the other programmer and I kept fixing the specific bugs—fixing symptoms—in order to retain a flexibility that nobody needed.

When you implement features in your projects, make them easy to use; don’t make them unnecessarily flexible. There is a difference.

Don’t allow unnecessary flexibility.

PORTED CODE IS NEW CODE

One lesson I learned from porting so much Windows Excel code to Macintosh Excel was that there was a temptation to skimp on testing such code. After all, I reasoned, the code was tested in the original product…

I should have caught all of the bugs in the Excel number formatting code before the code ever reached the testing group, but I didn’t. Instead, I copied the code to the Macintosh version of Excel, made the necessary changes to hook the code into the project, and then casually tested the code to verify that I’d hooked it in correctly.

I didn’t thoroughly test the feature itself because I thought it had already been tested. That was a mistake, especially since Windows Excel was under development itself and this was still in the days when Microsoft groups postponed fixing bugs to the end of the product cycle.

It doesn’t matter how you implement your features—whether you design and implement them from scratch or leverage existing code from another project—you are still responsible for keeping bugs out of the code that you add to your project. The fact that Windows Excel had the same bugs didn’t make the bugs any less severe in Macintosh Excel. I got lazy, and it showed.

Thoroughly test all code that you add to your project, regardless of its origin.

“TRY” IS A FOUR-LETTER WORD

How many times have you said something like, “I can’t figure out how to…” and another programmer has answered, “Have you tried...?” It’s common to see that dialogue in one form another on StackOverflow or any other programmers’ forum on the Web. One programmer will post a message asking, “How do I hide the cursor so that it doesn’t obscure the display?” Another programmer will respond, “Try moving the cursor to an off-screen coordinate.” Yet another will suggest, “Try setting the cursor mask to zeros to indicate that none of the cursor pixels should be visible.” Still a third might say, “The cursor is just a bitmap, so try setting its width and height to zero.”

Try. Try. Try.

Admittedly, this is a silly example, but I’m sure you’ve seen such dialogues take place, whether it was in an online forum or in front of the office coffee maker. And more often than not, none of the things you’re supposed to try is the proper course to pursue. When somebody suggests you “try something,” they’re guessing! Their guess might even give you your intended result, but because it is a guess that happens to work, the next release of a library or operating system could break your code. Obviously, you don’t want that.

Remember this when reading programmers’ forums for ideas: when you’re told to “try” something, it means the poster doesn’t know the correct solution either. Even if their guess works, you should still do research to find the actual documented solution to your need. You may just find that there is a HideCursor() function that does exactly what you want. No guessing involved.

If you find yourself testing possible solutions to a problem, stop yourself, get on the web and read from expert sites that thoroughly document the language you are programming in or the operating system that you are programming for. No, it’s not as much fun as playing around with code, or as easy as asking everybody for something to try, but you’ll learn more about the programming language and the operating system that you’re working with.

Don’t keep “trying” solutions until you find one that works. Take the time to find the correct solution.

THE SACRED SCHEDULE

There are programmers who, when given a sizable feature to implement, will spend two weeks hunched over a keyboard writing code, never bothering to test their work. Other programmers will implement a dozen small features before stopping to check things out. There’s nothing wrong with those approaches provided that the programmers thoroughly test their code.

But do they?

Think about the case in which a programmer has five days in which to implement five features. Suppose that programmer has two choices: to implement and test the features one at a time or to implement all five features and then test all five. Which approach, in practice, do you think will more likely result in rock-solid code? I’ve seen both coding styles over the years, and with rare exceptions, programmers who test code as they go have fewer bugs. I can even tell you why that’s true.

Suppose the programmer uses all five days to implement the five features but then realizes that he or she doesn’t have much, if any, scheduled time remaining to thoroughly test the code. Do you think the programmer will take an extra day or two, slipping the schedule, to thoroughly test that code, or do you think the programmer might instead give the code a quick test to simply verify that it seems to work? The answer will depend upon the individual programmer and their work environment. But when it comes down to slipping the schedule, which is frowned upon at most companies, or cutting back on testing which often results in no negative feedback whatsoever, programmers naturally cut back on testing when pressed for time.

If you want to write solid, bug-free code, you simply cannot accept the idea that a feature is “finished” until it has also been thoroughly tested by the programmer who wrote the code. Who better to test the code?

In fact, as a project lead, I won’t allow programmers to implement any new features until all bugs reported against their existing code are fixed. This prevents sloppy programmers from inundating the project with features that don’t work correctly, and it allows thoughtful programmers to proceed with development. You must have the philosophy and mindset that your product has to be rock-solid at every step of development.

It’s not how many features you have implemented, it’s about how many rock-solid features you have implemented.

Write and test code in small chunks.

Always test your code, even if that means you slip your schedule.

WHAT’S IN A NAME?

In Chapter 5, I explained how C’s getchar function often tricks programmers into thinking that the function returns a char when it actually returns an int. In the same way, programmers often believe that the testing group is responsible for testing their code after they’ve written it. What else would a testing group be for? But despite what many programmers believe, the testing group’s job is not to test the programmers’ code.

It might be easier to understand the role the testing group plays if you look at the same process in another field: house construction. There, contractors do the work, and inspectors verify it. But inspectors don’t test the work. An electrician would never wire up a house and then leave without first turning on the power, testing the circuit breakers, testing that all the lights worked, and checking every outlet using a receptacle tester. The electrician would never think, “I don’t need to do any tests. If there’s a problem, the inspector will let me know.” As you can imagine, any electrician who thought that way would soon find it hard to get work.

The good reason that testers, like inspectors, are not responsible for testing the work is that they rarely have the necessary access, tools, or skills. Despite computer folklore to the contrary, testers simply cannot test your code better than you can. Can testers add assertions to catch bad data flow? Can they incorporate subsystem tests like the ones in Chapter 3 for C’s memory manager? Can they use a debugger to step through code one instruction at a time to check that every code path is executed and works as expected?

The sad truth is that programmers, who actually can test their code more effectively than their testing groups, often don’t.

The testing group plays a valuable part in the development process, but it’s just not the part that many programmers think it is. When testers examine a product, they look for flaws and holes in features, they verify that the product is backwards compatible with previous releases, they alert the development team to quirks and rough edges that, if smoothed, would improve the product, and they use the product in “real world” scenarios to make sure that new features are truly useful. And, yes, testers report any bugs they discover.

Here’s another way to look at it: Before a project is released to the general public, it is typically first distributed to beta testers outside of the company or organization. These beta testers use the code in real-world scenarios and give feedback, including bugs they encounter, to help the developers improve the release before it is mass distributed to the public.

Programmers would never expect their beta testers to be responsible for testing their code. That said, programmers regularly expect their alpha testers—their in-house testing team—who have exactly the same job as beta testers, to test their code so they don’t have to.

Why?

It’s all in the name.

In the software development field, we would probably all be better off if we renamed our testing groups to be our quality assurance groups. That would be a much more accurate description of the testing group’s vital role in the product development process.

As a programmer, if your goal is to write solid, bug-free code, you simply cannot push the job of testing your code onto your testing group—that’s not their job. That’s your job. If you haven’t already adopted this philosophy about writing solid code, I would encourage you to change that.

I sometimes get the question that “if programmers are responsible for thoroughly testing their own code, aren’t programmers and testers duplicating each other’s efforts?” The answer is No.

When programmers test code, they test it from the inside and move out. Testers start from the outside and move in.

Programmers start by testing each function, stepping through every line of code, verifying code and data flow. From there, they move one step outward to verify that each function works correctly with the other functions in its subsystem. Finally, programmers use unit tests to verify that the subsystems interact properly. The unit tests may, as extra verification, regularly check the state of internal data structures throughout the tests.

Testers, on the other hand, view code as a black box and write global tests that throw all possible inputs at the program as a way of looking for flaws. The testers may also use regression tests to verify that all reported bugs have been fixed and have stayed fixed. From there, testers steadily move inward, using code coverage tools to give them an idea of how much internal code their global tests are executing. The testers then use that information to create new tests that try to execute the untouched code.

This is a grand example of using two separate “algorithms” to test the program. The combination of approaches works because programmers focus on code while testers focus on features. With the two working from opposite directions, the chances of finding unknown bugs are increased.

Don’t rely on the testing group to find the bugs in your code. That’s your job.

TESTERS WEAR WHITE HATS

Have you ever noticed how some programmers heave a sigh of relief when the testing group finds a bug. “Whew!” they say, “I’m sure glad Testing found that bug before we shipped.” Other programmers resent it when a tester reports a bug, especially if the tester reports many bugs against the same body of code. I’ve seen programmers bristle with anger: “Why won’t that tester leave me alone?” I’ve heard project leads (who should know better) say, “it’s Testing’s fault that we’ve slipped this beta date.” Once, I even had to prevent a project lead and a testing lead from throwing punches at each other.

Does that sound silly? It’s easy for us to sit back and see how ridiculous such behavior is when we’re not the ones under pressure to ship a product and under attack. But when you’re months past your ship date and buried in bugs, it’s easy to view the testers as the Bad Guys.

When I see programmers getting upset with testers, I pull them aside and ask them why they’re holding the testers responsible for bugs that programmers created. It makes no sense to get angry at testers; they’re just the messengers.

When a tester reports a bug in your code, your first reaction should be shock and disbelief—you should never expect testers to find bugs in your code. Your second reaction should be gratitude because that tester has saved you from releasing buggy code.

Sometimes you’ll hear a programmer complain that a particular bug is ridiculous, or that certain testers regularly report silly bugs. If you hear a programmer grumbling about silly bugs, stop and remind him or her that it’s not up to testers to judge how serious bugs are or whether they’re worth fixing. Testers must report all bugs, silly or not, because for all they know, those silly bugs may be the side effects of serious problems.

The real question isn’t whether the bug is silly but why the programmer who tested the code didn’t catch the bug. Even if the bug is minor, it’s still important to determine its cause so that you can prevent similar bugs in the future.

A bug may be minor; that it exists is serious.

Don’t blame testers for finding your bugs.

WHAT ARE YOUR CODING PRIORITIES?

If you flip back through the pages of this book and look at all of the Quick Review points, you might be surprised to find that some of them appear to contradict others. Then again, maybe you won’t be surprised. After all, programmers have long dealt with the sometimes contradictory goals of writing fast code and writing tight code.

When you’re faced with a choice between two possible implementations, which do you choose? I doubt you would have trouble choosing between a fast algorithm and a small one—you make that choice all the time—but what about choosing between a fast algorithm and a maintainable one, or between a small-but-risky algorithm and a larger-but-easily-tested one? Some programmers could answer those questions without much thought, but others would be unsure, and if you asked them the same questions weeks apart, you might get different answers.

Those programmers would be unsure about such trade-offs because they don’t know what their coding priorities are beyond the common ones such as size and speed. Programming without a clear set of priorities is like going on a trip without knowing your destination. At every corner you have to stop and ask, “What do I do now?” and you’re bound to take wrong turns.

Then there are programmers who know quite clearly what their priorities are but who, because their priorities are wrong or conflicting, don’t ponder at corners, but instead consistently take wrong turns.

For example, many programmers today are still locked into the microprocessor priorities of the 70s and 80s where memory was so scarce that 64K of RAM was a fully loaded machine and microprocessors were so slow that a pong game taxed them. To write a usable program, you couldn’t even think about maintainability. Size and speed were the only factors programmers focused on. Back then, every programmer had a CPU reference manual on their desk describing every assembly language instruction, how many cycles each instruction required, and how many bytes of RAM each instruction needed. Programming in a high-level language such as C, Pascal, or FORTRAN was considered extraordinarily wasteful because the compilers generated such inefficient code compared to what you could hand-code directly in assembly language.

But today, programs are far more complex. RAM is measured in billions of bytes not in thousands of bytes. Microprocessors are so blindingly fast that even the most inefficient algorithms run without noticeable delay.

Hardware resources have changed dramatically, but the mindset about programming priorities has been slow to change along with those hardware changes. Size and speed continue to be the top priorities for programmers. Code testability, maintainability, and reusability should have a much higher priority than they currently do.

There’s no reason to employ all the size and speed tricks that were developed decades ago when resources were so tight. Focus instead on writing straightforward code that is easy to maintain and easy to test.

Will the code be slower or fatter? A little bit, perhaps. But what does a billionth of a second longer runtime mean to the end user? And what does a few extra bytes of compiled code matter when your end-users have billions of bytes of RAM in their machines?

Am I suggesting that it’s okay to write sloppy, ill-thought-out code because the hardware is so much more powerful today? Absolutely not.

I’m simply pointing out that the priorities have shifted. It simply doesn’t make sense to continue using all the size and speed tricks that were developed decades ago when it is now far more important that code is maintainable and testable.

If you’ve never thought about your coding priorities, or you haven’t reviewed them lately, it’s important that you sit down and consciously create a road map so that you can consistently make the best choices to accomplish the goals of your project. Note what I said there, “the goals of your project.” Your priority list should not reflect what you want to do, but what you should do.

Establish your coding priorities and stick to them.

YOU DON’T GET WHAT YOU DON’T ASK FOR

One important point that I haven’t yet mentioned in this chapter is that you must develop the habit of asking questions about how you code. This entire book is the result of consistently asking a few simple questions over a long period of time:

◆

How could I have automatically detected this bug?

◆

How could I have prevented this bug?

And in this last chapter:

◆

Does a belief I hold help or hinder my ability to write bug-free code?

It is important to examine your personal priority list. If you believe that the testing group exists to test your code, you’re going to have trouble writing bug-free code.

If you want to write bug-free code, you need to weed out the beliefs that prevent you from achieving that goal. Examine your beliefs and ask yourself whether this or that belief helps or hinders your ability to write bug-free code.

QUICK REVIEW

◆

Bugs don’t create themselves; nor do they fix themselves. If you have a bug report that you can’t reproduce, don’t assume that the tester was imagining things. Make the effort to find the bug, even if that means reverting to an older version of the program.

◆

Don’t fix bugs “later.” It’s becoming alarmingly common to read of major products being canceled because of runaway bug lists. Your project won’t suffer that fate if you fix bugs as you find them. You can’t have a runaway bug list if the project is always near zero bugs.

◆

When you track down a bug, always ask yourself whether the bug is a symptom of a larger problem. Yes, it’s easier to fix the symptom that you’ve tracked down, but you should always make the effort to find the true cause.

◆

Don’t write unnecessary code or make unnecessary code cleanups. Let your competitors implement cool but worthless features, clean up code, and slip their ship dates because of “free” features. Let your competitors waste time fixing the unnecessary bugs that come with all that useless code.

◆

Remember that being flexible is not the same as being easy to use. When you design functions and features, focus on making them easy to use. If your functions are merely flexible—as the realloc function and that color-formatting feature in Microsoft Excel are—you’re not making them more useful; you’re simply making it more difficult to find any bugs in the code.

◆

Resist the urge to “try” things to achieve a desired effect. Use the time you would spend trying different approaches to achieve your desired result and instead invest that time into researching the proper solution for what you need to accomplish. If you have to, contact Developer Support for the operating system that you’re coding for. They should be able to tell you exactly how to do what you want to do.

◆

Write your code in chunks small enough to thoroughly test, and don’t skimp on testing. If you don’t test your code yourself, it’s possible that nobody will. Whatever you do, don’t expect the testing group to test your code for you. That’s not their job.

◆

Determine what the coding priorities are for your project and make them clear to your team. If you’re not the project lead, start a conversation with your project lead about this important topic.

Epilogue

WHERE DO YOU

GO FROM HERE?

You may be wondering, “Steve, do you truly believe that it’s possible to write bug-free programs?” Absolutely I do for small programs that are only a few tens of thousands of lines of code. For the massive applications that I’ve worked on that span hundreds of thousands and millions of lines of code, that’s a different story. When you bring in whole casts of programmers, each of whom is making their own unique assumptions about how code should be written and using their own coding styles, it’s much more difficult to write completely bug-free code.

But even for massive projects, I do believe that you can come very close to writing bug-free programs, much closer than most programming teams achieve today. But those teams have to adopt the “zero bugs” philosophy for writing rock solid bug-free code, meaning that their projects should have no critical bugs at every step of development.

What do I mean by critical bugs? A critical bug is one that everyday, normal users will encounter on a regular basis. That kind of bug must be fixed, pronto.

Okay then, what’s a non-critical bug?

Here’s an example of a non-critical bug: Let’s say you have a web form that asks for a person’s first name, among other things. They enter their first name plus all the other data in the web form, and your code works absolutely flawlessly unless someone enters a first name that is more than 256 characters long.

I would call that a non-critical bug because no normal user would ever enter a first name with more than 256 characters. If the code truncates the name to the first 256 characters and stores that truncated result and not the full entry in your database, I wouldn’t give that bug any attention whatsoever. It’s not a real-world bug.

The reality is that you will never completely eliminate bugs, but you can increase the time between occurrences by constantly striving to abolish every class of bug you encounter. To help you in this endeavor, I’ve provided a task-oriented programmer’s checklist in Appendix A that covers the most important guidelines in this book.

The final key to successfully writing bug-free code can be summed up with one final guideline:

Never allow the same type of bug to bite you twice.

A

CODING

CHECKLISTS

To remind you of the most important points in the book, I've created several checklists you can review during the primary development steps: design, orientation, DEBUG support, testing, and debugging. I haven't listed the points that have to do with development overall—I assume that you're using your optional IDE or compiler warnings, maintaining debug versions of your program, fixing bugs as they are reported, and so on.

To make effective use of these checklists, review them regularly as you add new code to your project, especially if that code introduces new bugs.

DESIGN

When you consider different designs for a feature, don't stop with the design that gives you the fastest or the smallest result. Consider the risks involved in implementing, maintaining, and using the code that would result from your design. For each possible design, review these points.

◆

Does this design include undefined or meaningless behavior? What about random or rare behavior? Does the design allow unnecessary flexibility or make unnecessary assumptions? Are there arbitrary details in the design?

◆

Do you pass any data in global variables or global buffers? Do any functions rely on the internal workings of other functions? Do any functions do more than one task?

◆

Does your design have to handle any special cases? Have you isolated the code that handles those special cases?

◆

Look at the inputs and outputs of your functions. Does each of the inputs and outputs represent exactly one type of data, or do some of them contain error values or other hard-to-notice values? Robust interfaces make every input and every output explicit so that programmers can't miss important details such as the NULL error value returned by malloc, or the fact that realloc can release a memory block if you pass in a size of 0.

◆

Anticipate how programmers will call your functions. Does the “obvious” approach work correctly? Recall that in realloc’s case, the obvious approach creates lost memory blocks.

◆

On the maintenance side, are your functions readable at the point of call? Each function should perform exactly one task, and its arguments should make the meaning of the call clear. The presence of TRUE and FALSE arguments often indicates that a function is doing more than one task, or that is not well designed.

◆

Do any of your functions return error values? Is it possible to redefine those functions to eliminate those error conditions? Remember that when a function returns an error, that error must be handled—or mishandled—at every point of call.

◆

Most important, is it possible to automatically and thoroughly validate your design using DEBUG code or a unit test? If not, you should consider using an alternative design that can be tested.

IMPLEMENTATION

After implementing your design, you should review these points to ensure that your implementation is robust and error resistant.

◆

Compare your implementation to your design. Have you accurately implemented the design? Be careful. Minor differences between your design and implementation can trip you up. Remember the IntToStr example that broke because it used signed integers when the design called for unsigned integers.

◆

Examine the expressions in your code. Can any of them overflow or underflow? What about your variables?

◆

Have you used nested ?: operators or other risky language idioms such as shifting to divide or multiply? Have you mixed bitwise operators and arithmetic operators without good cause? Have you used any language idioms in a questionable way? For example, using the 0/1 result of a logical expression in an arithmetic context. Rewrite risky expressions using comparable yet safer expressions.

◆

Take a close look at your code. Have you used any arcane language idioms that the average programmer on your team wouldn't understand? Consider rewriting the code.

◆

Each of your functions probably performs a single task, but is that task implemented using a single code path, or is the task implemented with lots of special-case code? Can you eliminate those special cases by using an alternative algorithm? Try to eliminate every if statement in your code.

◆

Do you call any functions that return errors? Can you alter your design so you can minimize the number of times you have to call such functions?

◆

Do you reference memory you have no right to touch? Specifically, do you reference memory that you've released? Do you peek at private data structures owned by other subsystems?

◆

If your functions take pointers to inputs or to outputs, does your code restrict its references to only the memory required to hold those inputs and outputs? If not, your code may be making an erroneous assumption about how much memory the caller has allocated for that data.

ADDING DEBUG SUPPORT

Adding assertions and other debugging code to your implementations can dramatically reduce the time required to find bugs hiding in your code. This checklist points out worthwhile assertions and debugging code that you should consider using.

◆

Have you used assertions to validate your function arguments? If you find that you can't validate a particular argument because you don't have enough information, would maintaining extra debug information help? Recall how the DEBUG-only sizeofBlock function was useful in validating pointers to allocated memory.

◆

Have you used assertions to validate your assumptions or to detect illegal uses of undefined behavior? Asserting for undefined behavior prevents programmers from abusing unspecified details of your implementations.

◆

Defensive programming “fixes” internal bugs when they occur, making such bugs hard to spot. Have you used assertions to detect these bugs in the DEBUG version of your program? Of course, this view of defensive programming doesn't apply to defensive programming used to correct bad user input—bad user input is not a bug. You always need to handle bad user input.

◆

Are your assertions clear? If not, be sure to include comments to explain the tests. Unfortunately, when programmers get an assertion failure and don't understand the purpose of the test, they will often assume that the assertion is invalid and remove it. Comments help preserve your assertions.

◆

If your code allocates memory, have you used DEBUG -only code to set the uninitialized contents to a known but obviously garbage state? Setting memory to a consistent value will make it easier to find and reliably reproduce bugs that use uninitialized memory. Don’t use 0 as your fill value; use an oddball looking value that you’ll easily recognize as uninitialized memory.

◆

If your code releases memory, does your DEBUG code first destroy the contents so that you don't have valid–looking garbage hanging around?

◆

Are any of your algorithms critical enough that you should use a second, but different, DEBUG–only algorithm to verify the primary one?

◆

Are there any DEBUG checks you can make at program startup to detect bugs at the earliest possible moment? In particular, are there any data tables that you could validate at program startup?

◆

Is your target hardware fast enough that you can ship your final project with the DEBUG checks in place? If so, you could alert the end-user that “An internal error has occurred. Would you like to send an automated report to the development team so they can correct this issue in future versions? Yes/No.” Your code could then notify you of the assertion failure via the Internet. Modern desktop computers are so blindingly fast that it’s becoming common to ship with all DEBUG code fully in place. Use crowdsourcing to help find bugs if you can.

TESTING

It is vitally important that programmers test their code, even if it means slipping the schedule. The questions in this section point out the most beneficial steps to take.

◆

Does the code compile without generating any warnings when you have all optional compiler warnings turned on? If you’re using your particular language’s version of lint, or a similar diagnostic tool that looks at your code and warns you about anything questionable, does your code come up warning-free? Does your code pass all your unit tests? Do you even have unit tests? These are all automated debug-tools. If you aren’t using these tools, you are missing out on a major opportunity to easily detect bugs.

◆

Do you step through all new code using a debugger, focusing not only on the code, but also on the data as it flows through your code? This is perhaps the most effective approach to quickly catching bugs in your implementations.

◆

Do you clean up code that is already working perfectly fine? If so, do you re-test the code that you clean up? Remember, code that you clean up is new code that must be thoroughly tested. If you’re not willing to re-test your cleanups, don’t change code that is already working fine.

DEBUGGING

You should review the questions here each time you have to track down a reported bug.

◆

Were you unable to find a bug that was reported? If not, remember that bugs don't fix themselves. If you can’t reproduce a bug in the current version of your project, revert to the version that the bug was reported against.

◆

Have you found the true cause of the bug or merely a symptom of the bug? Track down the true cause of the bug.

◆

Ask yourself, “How could this bug have been prevented?” Come up with a precise guideline that could prevent this, or similar, bugs in the future.

◆

How could this bug have been detected automatically? Would an assertion have caught it? What about DEBUG code? What changes in your coding practices or process would help?

B

MEMORY

LOGGING

ROUTINES

The code in this appendix implements a simple linked-list version of the memory logging routines that are discussed in Chapter 3. The code is intentionally simple so that it can be easily understood—it is not meant to be used in any application that makes heavy use of the memory manager. But before you spend time rewriting the code to use an AVL-tree, a B-tree, or any other data structure that provides fast searches, first try the code to verify that it is indeed too slow for practical use in your application. You may find that the code works well for you, particularly if you don’t maintain many globally allocated memory blocks.

The implementation in this file is straightforward: For every allocated memory block, these functions allocate a small additional block of memory to hold a blockinfo structure that contains the log information. When a new blockinfo structure is created, it is initialized with the appropriate data and placed at the head of the linked-list structure.

Quick note: the following code assumes that we’re working on hardware that uses flat pointers. If that doesn’t describe your hardware, you may have to change this code. The key point is to understand the idea behind the code so that you can use similar thinking when beefing up your own subsystems.

block.h:

#ifdef DEBUG

/*----------------------------------------------------------

*  blockinfo is a structure that contains the memory log

*  information for one allocated memory block. Every

*  allocated memory block has a corresponding blockinfo

*  structure in the memory log.

*/

typedef struct BLOCKINFO

{

struct BLOCKINFO *pbiNext;

byte   *pb;                  /*  Start of block  */

size_t size;                 /*  Length of block  */

flag   fReferenced;          /*  Ever referenced? */

} blockinfo;                     /*  Naming: bi, *pbi. */

flag fCreateBlockInfo(byte *pbNew, size_t sizeNew);

void FreeBlockInfo(byte *pbToFree);

void UpdateBlockInfo(byte *pbOld, byte *pbNew, size_t sizeNew);

size_t sizeofBlock(byte *pb);

void ClearMemoryRefs(void);

void NoteMemoryRef(void *pv);

void CheckMemoryRefs(void);

flag fValidPointer(void *pv, size_t size);

#endif

block.c:

#ifdef DEBUG

/*----------------------------------------------------------

*  The functions in this file must compare arbitrary

*  pointers, an operation that the ANSI C standard does

*  not guarantee to be portable.

*

*  The macros below isolate the pointer comparisons needed

*  in this file. This implementation assumes "flat" pointers,

*  for which straightforward comparisons will always work.

*  The definitions below will *not* work for all processor

*  architectures.

*/

#define fPtrLess(pLeft, pRight)   ((pLeft) < (pRight))

#define fPtrGrtr(pLeft, pRight)   ((pLeft) > (pRight))

#define fPtrEqual(pLeft, pRight)  ((pLeft) == (pRight))

#define fPtrLessEq(pLeft, pRight) ((pLeft) <= (pRight))

#define fPtrGrtrEq(pLeft, pRight) ((pLeft) >= (pRight))

/*--------------------------------------------------------*/

/*       * * * * * Private data/functions * * * * *       */

/*--------------------------------------------------------*/

/*----------------------------------------------------------

*  The variable pbiHead points to a singly-linked list of

*  of debugging information for the memory manager.

*/

static blockinfo *pbiHead = NULL;

/*----------------------------------------------------------

*  pbiGetBlockInfo(pb)

*

*  pbiGetBlockInfo searches the memory log to find the

*  block that pb points into and returns a pointer to the

*  corresponding blockinfo structure of the memory log. Note:

*  pb *must* point into an allocated block or you'll get an

*  assertion failure; the function either asserts or succeeds

*  --it never returns an error.

*

*  blockinfo *pbi;

*  ...

*  pbi = pbiGetBlockInfo(pb);

*      // pbi->pb points to the start of pb's block

*      // pbi->size the size of the block

*/

static blockinfo *pbiGetBlockInfo(byte *pb)

{

blockinfo *pbi;

for (pbi = pbiHead; pbi != NULL; pbi = pbi->pbiNext)

{

byte *pbStart = pbi->pb;

byte *pbEnd = pbi->pb + pbi->size - 1;

if (fPtrGrtrEq(pb, pbStart)  &&  fPtrLessEq(pb, pbEnd))

break;

}

/*  Couldn't find pointer? Is it (a) garbage? (b) pointing

*  to a block that was freed? Or (c) pointing to a block

*  that moved when it was resized by fResizeMemory?

*/

ASSERT(pbi != NULL);

return (pbi);

}

/*----------------------------------------------------------

*  fCreateBlockInfo(pbNew, sizeNew)

*

*  This function creates a log entry for the memory block

*  defined by pbNew:sizeNew. The function returns TRUE if it

*  successfully creates the log information; FALSE otherwise.

*

*  if (fCreateBlockInfo(pbNew, sizeNew))

*      // success -- the memory log has an entry

*  else

*      // failure -- no entry, so release pbNew

*/

flag fCreateBlockInfo(byte *pbNew, size_t sizeNew)

{

blockinfo *pbi;

ASSERT(pbNew != NULL  &&  sizeNew != Ø);

pbi = (blockinfo *)malloc(sizeof(blockinfo));

if (pbi != NULL)

{

pbi->pb = pbNew;

pbi->size = sizeNew;

pbi->pbiNext = pbiHead;

pbiHead = pbi;

}

return (flag)(pbi != NULL);

}

/*----------------------------------------------------------

*  FreeBlockInfo(pbToFree)

*

*  This function destroys the log entry for the memory block

*  that pbToFree points to. pbToFree *must* point to the start

*  of an allocated block or you will get an assertion failure.

*/

void FreeBlockInfo(byte *pbToFree)

{

blockinfo *pbi, *pbiPrev;

pbiPrev = NULL;

for (pbi = pbiHead; pbi != NULL; pbi = pbi->pbiNext)

{

if (fPtrEqual(pbi->pb, pbToFree))

{

if (pbiPrev == NULL)

pbiHead = pbi->pbiNext;

else

pbiPrev->pbiNext = pbi->pbiNext;

break;

}

pbiPrev = pbi;

}

/*  if pbi is NULL then pbToFree is invalid  */

ASSERT(pbi != NULL);

/*  Destroy the contents of *pbi before freeing them.  */

memset(pbi, bGarbage, sizeof(blockinfo));

free(pbi);

}

/*----------------------------------------------------------

* UpdateBlockInfo(pbOld, pbNew, sizeNew)

*

* UpdateBlockInfo looks up the log information for the memory

* block that pbOld points to. The function then updates the

* log information to reflect the fact that the block now lives

* at pbNew and is “sizeNew bytes” long. pbOld *must* point to

* the start of an allocated block; otherwise you will get an

* assertion failure.

*/

void UpdateBlockInfo(byte *pbOld, byte *pbNew, size_t sizeNew)

{

blockinfo *pbi;

ASSERT(pbNew !=NULL  &&  sizeNew != Ø);

pbi = pbiGetBlockInfo(pbOld);

ASSERT(pbOld == pbi->pb);     /*  Must point to start  */

pbi->pb = pbNew;

pbi->size = sizeNew;

}

/*----------------------------------------------------------

*  sizeofBlock(pb)

*

*  sizeofBlock returns the size of the block that pb points

*  to. pb *must* point to the start of an allocated block

*  or you will get an assertion failure.

*/

size_t sizeofBlock(byte *pb)

{

blockinfo *pbi;

pbi = pbiGetBlockInfo(pb);

ASSERT(pb == pbi->pb);   /*  Must point to start  */

return (pbi->size);

}

/*-----------------------------------------------------*/

/*  The following routines are used to find dangling   */

/*  pointers and lost memory blocks. See Chapter 3     */

/*  for a discussion of these routines.                */

/*-----------------------------------------------------*/

/*----------------------------------------------------------

*  ClearMemoryRefs(void)

*

*  ClearMemoryRefs marks all blocks in the memory log as

*  being unreferenced.

*/

void ClearMemoryRefs(void)

{

blockinfo *pbi;

for (pbi = pbiHead; pbi != NULL; pbi = pbi->pbiNext)

pbi->fReferenced = FALSE;

}

/*----------------------------------------------------------

*  NoteMemoryRef(pv)

*

*  NoteMemoryRef marks the block that pv points into as

*  being referenced. Note: pv does *not* have to point to

*  the start of a block; it may point anywhere within an

*  allocated block.

*/

void NoteMemoryRef(void *pv)

{

blockinfo *pbi;

pbi = pbiGetBlockInfo((byte *)pv);

pbi->fReferenced = TRUE;

}

/*----------------------------------------------------------

*  CheckMemoryRefs(void)

*

*  CheckMemoryRefs scans the memory log looking for blocks

*  that have not been marked with a call to NoteMemoryRef.

*  If this function finds an unmarked block, it asserts.

*/

void CheckMemoryRefs(void)

{

blockinfo *pbi;

for (pbi = pbiHead; pbi != NULL; pbi = pbi->pbiNext)

{

/*  A simple check for block integrity. If this

*  assert fires, it means something is wrong with

*  the debug code that manages blockinfo or,

*  possibly, that a wild memory store has trashed

*  the data structure. Either way, there’s a bug.

*/

ASSERT(pbi->pb != NULL  &&  pbi->size != Ø);

/*  Check for lostv or leaky memory. If this assert

*  fires, it means the app has either lost track

*  of this block, or not all global pointers

*  have been accounted for with NoteMemoryRef.

*/

ASSERT(pbi->fReferenced);

}

}

/*----------------------------------------------------------

*  fValidPointer(pv, size)

*

*  fValidPointer verifies that pv points into an allocated

*  memory block and that there are at least "size" allocated

*  bytes frompv to the end of the block. If either condition

*  is not met,fValidPointer will assert; the function will

*  never return FALSE.

*

*  The reason fValidPointer returns a flag at all (always

*  TRUE) is to allow you to call the function within an

*  ASSERT macro. While this isn't the most efficient method

*  to use, using the macro neatly handles the debug-vs.-ship

*  version control  without having to resort to #ifdef

*  DEBUG's or to introduce other ASSERT-like macros.

*

*  ASSERT(fValidPointer(pb, size));

*/

flag fValidPointer(void *pv, size_t size)

{

blockinfo *pbi;

byte *pb = (byte *)pv;

ASSERT(pv != NULL  &&  size != Ø);

pbi = pbiGetBlockInfo(pb); /*  This validates pv  */

/*  size isn’t valid if pb+size overflows the block  */

ASSERT(fPtrLessEq(pb + size, pbi->pb + pbi->size));

return (TRUE);

}

#endif

C

ANSWERS

This appendix contains the answers to all of the questions in the “Things to Think About” sections in the book. Note that the open-ended PROJECT ideas aren’t treated in this appendix.

CHAPTER 1

1.

Your IDE or lint catches the precedence bug because it interprets the expression as:

while (ch = (getchar() != EOF))

In other words, the tool sees an expression being assigned to ch and assumes that you’ve mistyped == as =, and warns of the possible assignment bug.

2a.

The simplest way to catch accidental “octal bugs” is to enable your development environment to generate a warning anytime it sees an octal constant in your code. The work around: use decimal or hexadecimal constants instead. Both Java and C# have dropped support for octal constants, for good reason.

2b.

To catch cases in which programmers mistype & for && (or | for ||), your development environment could apply the same test that it uses to catch the case in which you mistype = for ==. Your development environment would generate an error when you used & (or |) in an if statement or compound conditional without explicitly comparing the result against 0. For example, this code would generate a warning:

if (u & 1)           /*  Is u an odd number? */

but this would not:

if ((u & 1) != Ø)    /*  Is u an odd number? */

2c.

The simplest method to warn of unintentional comments is to have your development environment issue a warning whenever the first character in a comment is either a letter of the alphabet or an opening parenthesis. Such a test would catch the two questionable cases:

quot=numer/*pdenom;

quot=numer/*(pointer expression);

To silence the warning, you would make your intentions clear by separating the / and the * with either a space or an opening parenthesis:

quot = numer / *pdenom;

quot=numer/(*pdenom);

/*But note: this comment would generate a warning*/

/*  This comment does not because of the leading space. */

/*-----------Nor does this comment-----------*/

2d.

You could have your development environment detect possible precedence errors by having it look for troublesome operator pairs that are used in the same, unparenthesized expression. For example, C programmers regularly introduce precedence bugs when they use the << and + operators in the same expression. Your development environment should issue a warning for code that mixes bitwise operators with arithmetic operators, without using parentheses:

word = bHigh << 8 + bLow;

This code is incorrect even though it looks valid. Your development environment should automatically alert you to this bug. With proper parentheses, you should not get a warning:

word = (bHigh << 8) + bLow;

word = bHigh << (8 + bLow);

A less ad hoc approach would be to use a heuristic such as “If the two operators have different precedence and they are not parenthesized, then issue a warning.” That heuristic is too simple to be practical, but you get the idea. Developing a good heuristic would require running a lot of code through your compiler or interpreter and tweaking the heuristic until you get useful results. You certainly wouldn’t want to get warnings for these common expressions:

word = bHigh*256 + bLow;

if (ch == ' '  ||  ch == '\t'  ||  ch == '\n')

3.

Your development environment could alert you to possible dangling- else clauses by issuing a warning whenever it encountered two consecutive unbracketed if statements followed by an unbracketed else statement:

if (expression1)            if (expression1)

if (expression2)            if (expression2)

⋮                           ⋮

else                            else

⋮                               ⋮

To silence the warning, you would use braces around the inner if statement:

if (expression1)            if (expression1)

{                           {

if (expression2)            if (expression2)

⋮                           ⋮

}                               else

else                                ⋮

⋮                       }

A good rule of thumb for writing rock-solid, bug-free code is to always use braces for your if and else clauses, even when they’re not needed.

4.

Putting constants and expressions on the left-hand side of your comparisons is useful because it provides one more method of automatically detecting bugs, but unfortunately, it works only when one of the operands is a constant or an expression—the technique is worthless if both operands are variables. Another problem with this method is that programmers must learn and remember to use the technique as they write code.

If, on the other hand, you use the optional warning switch in your development environment, your development environment would alert you to every possible assignment bug. Even better, the switch would work for programmers straight out of CS 101 who’ve never learned the benefits of reversing operands in their comparisons.

CHAPTER 2

1.

One possible implementation of the ASSERTMSG macro could have the macro take both an expression to validate and a string to display if the assertion fails. For example, to print the memcpy message, you would call ASSERTMSG this way:

ASSERTMSG(pbTo >= pbFrom+size  ||  pbFrom >= pbTo+size,

"memcpy: the blocks overlap");

In the implementation of the ASSERTMSG macro that follows, you would put ASSERTMSG’s definition in a header file and the _AssertMsg routine in a convenient source file.

#ifdef DEBUG

void _AssertMsg(char *strMessage); /*  prototype  */

#define ASSERTMSG(f,str) if (f) {} else _AssertMsg(str)

#else

#define ASSERTMSG(f,str)

#endif

And here’s the routine in another file:

#ifdef DEBUG

void _AssertMsg(char *strMessage)

{

fflush(NULL);

fprintf(stderr, "\n\nAssertion failure in %s\n",

strMessage);

fflush(stderr);

abort();

}

#endif

2.

The simplest way to detect bugs in switch statements that have not been updated is to include assertions in the default cases to alert you to unexpected cases that pop up. In some instances the default case should never be invoked because all of the possible cases are explicitly handled. When all cases are handled explicitly, use

⋮

default:

ASSERT(FALSE);   /*  We should never get here. */

break;

}

3.

By design, the pattern for each entry in the table must be a subset of the corresponding mask. For example, if the mask is 0xFF00, the pattern must not have any bits set in the low byte; otherwise, it would be impossible for any instruction, once masked, to match the pattern. The CheckIdInst routine could be enhanced to verify that the pattern is a subset of the mask:

void CheckIdInst(void)

{

identity *pid, *pidEarlier;

instruction inst;

for (pid = &idInst[Ø]; pid->mask != Ø; pid++)

{

/* Make sure pat is a subset of the mask. */

ASSERT((pid->pat & pid->mask) == pid->pat);

⋮

4.

Use assertions to verify that inst has none of the problematic settings:

instruction *pcDecodeEOR(instruction inst, instruction *pc, opcode *popc)

{

/* Did we get a CMPM or CMPA.L by mistake? */

ASSERT(eamode(inst) != 1  &&  mode(inst) != 3);

/*  If non-register mode, only allow absolute word

*  and long modes

*/

ASSERT(eamode(inst) != 7  ||  (eareg(inst) == Ø  ||

eareg(inst) == 1));

⋮

5.

The important point in choosing a backup algorithm is that it be a different algorithm. To verify that qsort is working, you could scan the data after a sort to verify that the order is correct. Scanning is not sorting, and therefore qualifies as a different algorithm. To verify that the binary search is working, follow it with a linear scan to see if the two searches give the same result. Finally, to verify that the itoa function is working, take the string it returns, reconvert the string to an integer, and compare the value to the integer originally passed to itoa; they should be equal.

Of course, you probably don’t want to use back up algorithms for every piece of code you write—that is, unless you’re working on code for a jet airplane, a radiation machine, or any other device in which a coding bug could be life threatening. But you probably should be using back up algorithms for all of the major engines in your application.

CHAPTER 3

1.

You can make it easier to distinguish between code that uses uninitialized data and code that continues to use released data by using different debug values to destroy the two types of memory. For example, fNewMemory could destroy new, uninitialized memory using bNewGarbage, and FreeMemory could destroy the memory it releases using bFreeGarbage.

#define bNewGarbage   ØxA3

#define bFreeGarbage  ØxA5

fResizeMemory creates both types of garbage—you could use the two values above, or you could create two more values.

2.

One way to catch “overfill” bugs is to periodically check the bytes following each allocated block to verify that they have not been modified. But while that test sounds straightforward, it requires that you remember what all those trailing bytes are, and it ignores the potential problems you might run into by reading from memory that isn’t part of the allocated block. Fortunately, there is a simple way to implement the test, provided you’re willing to allocate 1 extra byte for every block that you allocate.

Here’s what I mean: When you call fNewMemory with a size of 36, fNewMemory could actually allocate 37 bytes and store a known “debugging byte” in the extra memory location. Similarly, you could allocate and set an extra byte in fResizeMemory when it calls realloc. To catch the overfill bugs, you would then put assertions in sizeofBlock, fValidPointer, FreeBlockInfo, NoteMemoryRef, and CheckMemoryRefs to verify that the debugging byte has not been touched.

One way you could implement the code is shown below. First, you would define bDebugByte and sizeofDebugByte:

/*  bDebugByte is a magic value that is stored at the

*  tail of every allocated memory block in DEBUG

*  versions of the program. sizeofDebugByte is added

*  to the sizes passed to malloc and realloc so that

*  the correct amount of space is allocated.

*/

#define bDebugByte ØxE1

#ifdef DEBUG

#define sizeofDebugByte 1

#else

#define sizeofDebugByte Ø

#endif

Next, you would use sizeofDebugByte to adjust the calls to malloc and realloc in fNewMemory and fResizeMemory, and you would use bDebugByte to fill in the extra bytes if the allocations are successful:

flag fNewMemory(void **ppv, size_t size)

{

byte **ppb = (byte **)ppv;

ASSERT(ppv != NULL  &&  size != Ø);

*ppb = (byte *)malloc(size + sizeofDebugByte);

#ifdef DEBUG

{

if (*ppb != NULL)

{

*(*ppb + size) = bDebugByte;

memset(*ppb, bGarbage, size);

⋮

flag fResizeMemory(void **ppv, size_t sizeNew)

{

byte **ppb = (byte **)ppv;

byte *pbNew;

⋮

pbNew = (byte *)realloc(*ppb, sizeNew + sizeofDebugByte);

if (pbNew != NULL)

{

#ifdef DEBUG

{

*(pbNew + sizeNew) = bDebugByte;

UpdateBlockInfo(*ppb, pbNew, sizeNew);

⋮

Finally, you would put the assertion below into the sizeofBlock, fValidPointer, FreeBlockInfo, NoteMemoryRef, and CheckMemoryRefs routines that are in Appendix B.

/*  Verify that nothing wrote off end of block.  */

ASSERT(*(pbi->pb + pbi->size) == bDebugByte);

With these changes, the memory subsystem would catch bugs in which code writes past the end of your allocated memory blocks.

3.

There are many ways you could catch the not-so-dangling pointer bug. One possible solution would be to change the debug version of FreeMemory so that it wouldn’t actually free the blocks it receives, but would instead build a list of free, but allocated blocks. (The blocks would look allocated to the system but would look free to your program). Modifying FreeMemory this way would keep a “free” block from being reallocated before the memory subsystem could be validated with a call to CheckMemoryRefs. CheckMemoryRefs would then validate the memory system and finish up by taking FreeMemory’s “free” list and releasing all the blocks.

Now, while this solution would catch not-so-dangling pointers, you probably shouldn’t use it unless your program suffers from such bugs. The reason: The solution violates the principle that DEBUG code is extra code, not different code.

4.

To validate the sizes of the objects that your pointers reference, you must consider two cases: pointers to entire blocks, and pointers to suballocations within blocks. For pointers to entire blocks, the strongest tests you can make are to verify that the pointers reference the starts of their blocks and that the block sizes match what the sizeofBlock function would return for them. For pointers to suballocations within blocks, the tests should be weaker: The pointer must point into a block, and the size must not reach beyond the tail end of the block.

So instead of using the existing NoteMemoryRef routine to mark both suballocations and complete blocks, you could use two functions to mark the two types of blocks. For suballocations, you could extend the existing NoteMemoryRef function by adding a size argument, and for marking full blocks, you could create a new NoteMemoryBlock function:

/*  NoteMemoryRef(pv, size)

*

*  NoteMemoryRef marks the block that pv points into

*  as being referenced. Note: pv does *not* have to

*  point to the start of a block; it may point anywhere

*  within an allocated block, but there must be at

*  least "size" bytes left in the block. For entire

*  blocks, use NoteMemoryBlock -- it provides stronger

*  validation.

*/

void NoteMemoryRef(void *pv, size_t size);

/*  NoteMemoryBlock(pv, size)

*

*  NoteMemoryBlock marks the block that pv points to as

*  being referenced. Note: pv *must* point to the start

*  of a block that is exactly "size" bytes long.

*/

void NoteMemoryBlock(void *pv, size_t size);

These two functions would let you catch the bugs posed in the question.

5.

To improve the integrity checks in Appendix B, you would first change the reference flag in the blockinfo structure to a reference count, and you would then update ClearMemoryRefs and NoteMemoryRef to handle the counter. That part is straightforward. The question, though, is how do you modify CheckMemoryRefs so that it will assert when some blocks have multiple references but doesn’t assert for other blocks?

One solution to this problem would be to enhance the NoteMemoryRef routine so that it would take a block ID tag in addition to the pointer to the block. NoteMemoryRef could then store the tag in the blockinfo structure, and CheckMemoryRefs could come along later and use the tag to verify the reference count. You can see the code to implement this change below. For header comments, see the original functions in Appendix B.

/*  blocktag is a list of all of the types of allocated

*  blocks maintained by the application. ClearMemoryRefs

*  sets all blocks to tagNone. NoteMemoryRef sets the

*  tag to a specific block type.

*/

typedef enum

{

tagNone,       /* Initially set to tagNone. */

tagSymName,

tagSymStruct,

tagListNode,   /* List nodes have two refs. */

⋮

} blocktag;

void ClearMemoryRefs(void)

{

blockinfo *pbi;

for (pbi = pbiHead; pbi != NULL; pbi = pbi->pbiNext)

{

pbi->nReferenced = Ø;

pbi->tag = tagNone;

}

}

void NoteMemoryRef(void *pv, blocktag tag)

{

blockinfo *pbi;

pbi = pbiGetBlockInfo((byte *)pv);

pbi->nReferenced++;

ASSERT(pbi->tag == tagNone  ||  pbi->tag == tag);

pbi->tag = tag;

}

void CheckMemoryRefs(void)

{

blockinfo *pbi;

for (pbi = pbiHead; pbi != NULL; pbi = pbi->pbiNext)

{

/*  A simple check for block integrity. If this

*  assert fires, it means something is wrong with

*  the debug code that manages blockinfo or,

*  possibly a wild memory store has trashed the

*  data structure. Either way, there’s a bug.

*/

ASSERT(pbi->pb != NULL  &&  pbi->size != Ø);

/*  A check for lost or leaky memory. If there are

*  no references at all, it means either that the

*  app has lost track of this block or that not

*  all global pointers are being accounted for

*  with NoteMemoryRef. Some types of blocks may

*  have more than one reference to them.

*/

switch (pbi->tag)

{

default:

ASSERT(pbi->nReferenced == 1);

break;

case tagListNode:

ASSERT(pbi->nReferenced == 2);

break;

⋮

}

}

}

6.

Desktop and mobile device developers normally test out-of-memory conditions by using a tool to allocate enormous amounts of memory so that the application’s normal memory requests start failing. That approach can work, but it is not very precise if you want to test a specific function to see how it handles low resource conditions. A better technique is to build an out-of-resource simulator directly into—in this case—the memory manager.

But notice, memory errors are just one type of resource failure—you can have disk errors, out-of-paper errors, loss of internet connection errors, all sorts of errors. What’s really needed is a general tool to fake failures.

One possible solution would be to create a failureinfo structure that would contain information to tell the failure mechanism what to do. The idea is that programmers and testers would fill in the failureinfo structure from an external test and then exercise the feature they’re trying to test. Microsoft applications often have debug-only dialogs that allow testers to use such systems, and in applications such as Excel that have macro languages, debug-only macros allow testers to automate the process.

To declare the failureinfo structure for the memory manager, you would use

failureinfo fiMemory;

To simulate out-of-memory errors in fNewMemory and fResizeMemory, you would insert a small block of debug code to each function:

flag fNewMemory(void **ppv, size_t size)

{

byte **ppb = (byte **)ppv;

#ifdef DEBUG         /*  Added debug code  */

if (fFakeFailure(&fiMemory))

{

*ppb = NULL;

return (FALSE);

}

#endif

⋮

flag fResizeMemory(void **ppv, size_t sizeNew)

{

byte **ppb = (byte **)ppv;

byte *pbNew;

#ifdef DEBUG

if (fFakeFailure(&fiMemory))

return (FALSE);

#endif

⋮

With these changes, the failure mechanism is in place. To make it work, you would call the SetFailures function to initialize the failureinfo structure:

SetFailures(&fiMemory, 5, 7);

Calling SetFailures with 5 and 7 tells the failure system that you want to successfully call the system five times before getting seven consecutive forced failures. Two common calls to SetFailures would be

/*  Don’t fake any failures  */

SetFailures(&fiMemory, UINT_MAX, Ø);

/*  Always fake failures */

SetFailures(&fiMemory, Ø, UINT_MAX);

Using SetFailures, you can write unit-tests that call the same code over and over but each time calling SetFailures with different values to simulate all possible error patterns. A common test is to hold the second “fail” value at UINT_MAX while the first “success” count is progressively bumped from 0—“always fail”—to some number that is deemed large to test each successful call to the system.

Finally, there are times when you will want to call the memory system, disk system, and so on, and you definitely won’t want any fake failures; this is often true when you’re allocating resources from within other debug code. The following two nestable functions allow you to temporarily disable the failure mechanism:

DisableFailures(&fiMemory);

/* Do some allocating... */

EnableFailures(&fiMemory);

The code below implements the four functions that make up the failure mechanism.

typedef struct

{

/* # of times to succeed before failing */

unsigned nSucceed;

/* # of times to fail after succeeding */

unsigned nFail;

/* # of times already called */

unsigned nTries;

/* if lock is > Ø, disable mechanism */

int lock;

} failureinfo;

void SetFailures(failureinfo *pfi, unsigned nSucceed, unsigned nFail)

{

/*  if nFail is Ø, nSucceed must be UINT_MAX  */

ASSERT(nFail != Ø  ||  nSucceed == UINT_MAX);

pfi->nSucceed = nSucceed;

pfi->nFail    = nFail;

pfi->nTries   = Ø;

pfi->lock     = Ø;

}

void EnableFailures(failureinfo *pfi)

{

ASSERT(pfi->lock > Ø);

pfi->lock--;

}

void DisableFailures(failureinfo *pfi)

{

ASSERT(pfi->lock >= Ø  &&  pfi->lock < INT_MAX);

pfi->lock++;

}

flag fFakeFailure(failureinfo *pfi)

{

ASSERT(pfi != NULL);

if (pfi->lock > Ø)

return (FALSE);

if (pfi->nTries != UINT_MAX)  /* Don't overflow */

pfi->nTries++;

if (pfi->nTries <= pfi->nSucceed)

return (FALSE);

if (pfi->nTries - pfi->nSucceed <= pfi->nFail)

return (TRUE);

return (FALSE);

}

CHAPTER 4

There were no questions in Chapter 4, although some projects were suggested.

CHAPTER 5

1.

strdup has a risky interface because its error return value is disguised as a NULL pointer where, like malloc’s, it can be overlooked. A less error-prone interface would separate the error condition from the pointer output to make the error condition obvious. One such interface would be:

char *strDup;     /*  pointer to the copied string  */

if (fStrDup(&strDup, strToCopy))

successful--strDup points to the new string

else

unsuccessful--strDup is NULL

2.

A better interface for getchar than fGetChar’s interface would be one that returns an error code instead of a TRUE or FALSE “success” value. For example:

/*  These are the errors that errGetChar may return.  */

typedef enum

{

errNone = Ø,

errEOF,

errBadRead,

⋮

} error;

void ReadSomeStuff(void)

{

char ch;

error err;

if ((err = errGetChar(&ch)) == errNone)

success -- ch has the next character

else

failure -- err has the error type

⋮

This interface is better than fGetChar’s interface because it allows errGetChar to return multiple error conditions. If you don’t care about the specific kind of error being returned, you can eliminate the local variable err and revert to fGetChar’s interface style:

if (errGetChar(&ch) == errNone)

success -- ch has the next character

else

failure -- we don’t care what type of error we have

3.

The trouble with strncpy is that it is inconsistent in its behavior: Sometimes strncpy terminates the destination string with a null character, and sometimes it doesn’t. strncpy is listed along with other general purpose string functions, and programmers may erroneously conclude that strncpy is itself a general-purpose string function. It isn’t. strncpy really shouldn’t be in the ANSI standard given its unusual behavior, but the function was included in the Standard because of its widespread usage in pre-ANSI implementations of C.

4.

C++’s inline function specifier is valuable because it allows you to define functions that are as efficient as macros, yet don’t have the troublesome side effects that C’s macro functions have in evaluating their parameters.

5.

The serious problem with C++’s & reference arguments is that such arguments hide the fact that you are passing the variable by reference, not by value, and that can cause confusion. For example, suppose you redefine the fResizeMemory function so that it uses a reference argument. Programmers could then write:

if (fResizeMemory(pb, sizeNew))

resize was successful

But notice, programmers unfamiliar with the function would have no reason to believe that pb might be changed during the call. How do think that will affect program maintenance?

A related concern is that C programmers often manipulate the formal arguments to their functions because they know those arguments are passed by value, not reference. But consider the maintenance programmer who fixes a bug in a function he didn’t write. If that programmer fails to notice the & in the declaration, he could modify the argument without realizing that the change won’t be local to the function. & reference arguments are risky because they hide an important implementation detail.

6.

The problem with strcmp’s interface is that its return value leads to unintelligible code at the point of call. To improve strcmp, you would design the interface so that the return value is easily understood, even by those unfamiliar with the function.

One possible interface is a minor variant on the one that strcmp already has. Instead of returning arbitrary negative and positive values for unequal strings—which forces programmers to make all their comparisons relative to 0—you could change strcmp so that it returns three well-defined named constants:

if (strcmp(strLeft, strRight) == STR_LESS)

if (strcmp(strLeft, strRight) == STR_GREATER)

if (strcmp(strLeft, strRight) == STR_EQUAL)

Another possible interface would be to use separate functions for each type of comparison:

if (fStrLess(strLeft, strRight))

if (fStrGreater(strLeft, strRight))

if (fStrEqual(strLeft, strRight))

CHAPTER 6

1.

If you declare wndDisplay as a global window structure, you give it a special attribute that no other window structure has: it is a global. That may seem like a minor detail, but it can introduce unexpected bugs. For instance, suppose you want to write a routine to free a window and all of its children. This function will do it:

void FreeWindowTree(window *pwndRoot)

{

if (pwndRoot != NULL)

{

window *pwnd, *pwndNext;

ASSERT(fValidWindow(pwndRoot));

for (pwnd = pwndRoot->pwndChild; pwnd != NULL; pwnd = pwndNext)

{

pwndNext = pwnd->pwndSibling;

FreeWindowTree(pwnd);

}

if (pwndRoot->strWndTitle != NULL)

FreeMemory(pwndRoot->strWndTitle);

FreeMemory(pwndRoot);

}

}

Now notice, if you want to free every window, you can safely pass pwndDisplay because it points to an allocated window structure, but you cannot pass &wndDisplay because the code will try to free wndDisplay, which is impossible because it is a global. To make the code work correctly with &wndDisplay, you would have to insert

if (pwndRoot != &wndDisplay)

before the call to FreeMemory. If you do that, you tie the code to a global data structure. Yuck.

One of the best ways to keep bugs out of your code is to keep arbitrary design quirks out of your implementations.

2.

The second version is much riskier than the first for several reasons. Because the A’s, D’s, and expression are common code in the first version, they are going to be executed—and therefore tested—no matter what the value of f is. In the second version, the A’s and D’s will be tested separately, and unless they’re identical, you risk missing bugs in one case or the other. It’s also possible that the A’s and D’s in each case would not be identical if they were optimized specifically for use with B or C. That’s problematic.

In the second version, you’ll also have problems keeping the A’s and D’s synchronized as programmers fix bugs and enhance the code. That’s particularly true if the A’s and D’s are not identical. My advice is to use the first version unless calculating f is so expensive that the user will notice the difference. Here’s another good rule to remember: Minimize the differences by maximizing the amount of common code.

3.

It is risky using similar names like s1 and s2 because it’s easy to type s1 when you mean s2. Worse, the code will compile without issuing an error. Using similar names also makes it harder to spot bugs where you have swapped the names by mistake:

int strcmp(const char *s1, const char *s2)

{

for ( ; *s1 == *s2; s1++, s2++)

{

if (*s1 == '\Ø')   /*  Match to the end? */

return (Ø);

}

return ((*(unsigned char *)s2 < *(unsigned char *)s1) ? -1 : 1);

}

The code above is wrong because the test in the return statement is backwards, but it’s hard to see the bug because the names have no meaning. If you use descriptive and distinct names such as sLeft and sRight, the odds of having either the mistyping or the swapping kind of bug drop dramatically, and the code is more readable too.

4.

The ANSI C standard guarantees that you can address the first byte following any declared array, but it does not guarantee that you can reference the byte that precedes such an array. Nor does the standard guarantee that you can address the byte preceding a block that you allocate using malloc.

For example, the pointers for some Intel processors are implemented using base:offset pairs where only the unsigned offsets are manipulated. If pchStart is such a pointer and it points to the start of an allocated block, its offset is 0. If you assume that pch starts out with a value of pchStart+size, pch can never be less than pchStart because its offset can never be less than pchStart’s offset of 0.

5a.

Using printf(str) instead of printf("%s", str) will cause bugs if str contains any % signs— printf will misinterpret those % signs as format specifications. The trouble with printf("%s", str) is that it can so “obviously” be optimized to printf(str) that unwary programmers will occasionally clean up the code and introduce bugs.

5b.

Using f = 1-f; instead of f = !f; is risky because it assumes that f is either 0 or 1, whereas using !f clearly shows that you are flipping a flag and works for all values of f. The only reason to use 1-f is that it may generate slightly more efficient code than !f, but remember, local efficiency improvements rarely have any overall affect on a program’s performance. Using 1-f merely increases your risk of having a bug.

5c.

The risk in using multiple assignments in one statement is that it may cause unexpected data conversions. In the example here, the programmer was careful to declare ch as an int so that it could properly handle the EOF value that getchar might return. But notice that getchar’s value is first stored in a string, which means the value is converted to a char, and it is the converted char—not the returned int—that is assigned to ch. This unexpected conversion re-introduces the getchar bug we covered in Chapter 5, despite the fact that ch was so carefully defined to be an int.

6.

In a typical case, a table simplifies code by making it smaller and faster, which increases the odds that the code will be correct. You get a more balanced view of this question, though, when you consider the data in the table. The code may be small, but the table takes memory, so overall, the table solution may use more memory than the nontable implementation. The other problem with a table is risk—you must ensure that the data in the table is correct. Sometimes that’s easy, as it is in the tolower and uCycleCheckBox tables, but in a large table like the one in the disassembler in Chapter 2, it would be easy for a bug to creep in. So here’s another guideline: Don’t use a table solution unless you can validate the data in the table . That’s why I was so fastidious in verifying the data in the table for the disassembler.

7.

If your compiler or interpreter doesn’t make such basic optimizations as converting multiplications and divisions into shifts—when appropriate—you must have far worse code generation problems to worry about. Don’t make minor efficiency tweaks to overcome the limitations of a poor development system.

8.

To guarantee that it’s always possible to save the user’s file, simply allocate the buffer sometime before the user changes the file. If you need one buffer per file, then allocate a buffer every time you open a file. If the allocation fails, open the file as a read-only document, or don’t open the file at all. But if you need just one buffer to handle all open files, then you could allocate that buffer during program initialization. Don’t worry about wasting memory by having that buffer hanging around most of the time doing nothing. It’s much better to waste that small amount of memory so you can guarantee that you can save the user’s data. That’s a much better solution than to let your end-user work for five hours and then fail to save the data because you can’t allocate the buffer.

CHAPTER 7

1.

The code below modifies pchTo and pchFrom, both of which are inputs to the function.

char *strcpy(char *pchTo, char *pchFrom)

{

char *pchStart = pchTo;

while (*pchTo++ = *pchFrom++)

{}

return (pchStart);

}

Modifying pchTo and pchFrom doesn’t violate the write privilege associated with those arguments because they are passed by value, which means that strcpy receives duplicates of the inputs, and strcpy is therefore allowed to change them. But note that not all computer languages—FORTRAN is one example—pass arguments by value. This practice is quite safe in C and C-based languages, but it can be hazardous if you use it with other languages.

2.

Using memset to initialize adjacent locals is extremely risky, completely ridiculous, and inefficient compared to using the straightforward

i = Ø;   /* Set i, j, k, to Ø */

j = Ø;

k = Ø;

or using the more terse

i = j = k = Ø;

These pieces of code are both portable and efficient, and they are so obvious that you don’t even need the comments. The memset version is another matter.

I’m not sure what the original programmer was trying to gain by using memset, but I’m sure he or she didn’t get a good return on the effort. For starters, the overhead alone of calling memset is more expensive than explicitly setting i, j, and k to zero. But let’s assume that the programmer was using a smart compiler that inlines small fills when the fill value and length are known at compile time. That doesn’t improve things much: The code still assumes that the compiler will allocate i, j, and k adjacently on the stack, with k lowest in memory. The code also assumes that i, j, and k abut each other without any extra “pad” bytes to align the variables for efficient access.

But who’s to say that the variables even have frame storage? Good compilers routinely perform life-span analysis and use the information to keep locals in registers for their entire lives. For example, i and j may be “allocated” in registers and spend their entire lives there, never getting frame storage. k, on the other hand, must be given frame storage because its address is passed to memset (you can’t take the address of a register). In this scenario, i and j would remain uninitialized, and 2*sizeof(int) bytes following k would be erroneously set to 0.

This is a real-world example of an absurd coding practice. Don’t hire programmers who write code like this.

3.

The problem with not passing val if it isn’t needed is that the caller is making an assumption about the internal workings of DoOperation in much the way FILL was making assumptions about CMOVE. Suppose that a programmer enhances DoOperation and in the process rewrites it so that it always references val:

void DoOperation(operation op, int val)

{

if (op < opPrimaryOps)

DoPrimaryOps(op, val);

else if (op < opFloatOps)

DoFloatOps(op, val);

else

⋮

What happens when DoOperation references the nonexistent val? That depends upon the programming language or operating system you’re using, but for natively compiled code like C, the code could abort if val is in a read-protected portion of the stack frame.

The situation isn’t nearly as serious—or serious at all—for languages where unpassed arguments are handled safely and set to a predetermined value. But in that case, is it documented that it’s okay to not pass val in some cases, or has the programmer peeked inside your code and just noticed that not passing val would work?

If necessary, you can make it difficult for programmers to play tricks with your functions by forcing them to pass placeholders for unused variables. In the documentation, you could say, “Pass 0 for val whenever you call DoOperation with opNegAcc.” A well-placed assertion would help keep programmers honest:

case opNegAcc:

ASSERT(val == Ø);   /*  Pass Ø for val  */

accumulator = -accumulator;

break;

4.

Although the assertion verifies that f is either 1 or 0, the test is unclear. First, to even decipher what’s going on, you need to notice that a bitwise “and” is being used and not the logical “and.” Second, code written like that is almost always the result of trying to get a micro-optimization in the generated machine code and such micro-optimizations are rarely worth the speed vs. comprehension trade-off anymore. That’s especially true in this case where the code will likely be stripped from the released version of the code. The test would be better written as:

ASSERT(f == 0  ||  f == 1);

CHAPTER 8

There were no exercises in Chapter 8.

D

THE INTERVIEW

QUESTION

In chapter 5, I mentioned that when I interview potential programmers to join our development team, I will often ask them to write a simple lowercase function that returns the lower-case character equivalent of the character passed to the function.

After the release of the first edition of Writing Solid Code, I was amazed by how many people asked me how having anybody write a function as simple as lowercase could demonstrate any coding skills at all. Although this question has nothing to do with the overall topic of this book, I thought I should address this often-asked question by walking you through a typical interview session.

As I go through the interview, you’ll see that I’m not that concerned with how the candidate implements the lowercase function. My primary concern is determining his or her thought processes—his or her philosophies about writing code.

In other words, How you think about writing code, matters.

After first reviewing the candidates’ resumé and answering any questions he or she might have, I hand them an ASCII table and ask them to write a lowercase function that returns the lowercase version of a character. On the surface, this problem might seem too simple to be an effective test of a programmer’s skills, but there’s no value in asking difficult—or worse—trick questions, which only serves to muddy the interview. If the candidate gets bogged down, is it because they don’t know how to program? Or, more possibly, are they simply immobilized because they don’t experience instantaneous insight under the pressure of an interview, in an unfamiliar office, wearing a suit, without the aid of a latte, where every second of silence feels like an eternity that exponentially makes them feel like they’re blowing the interview? Minds shut down with all that going on. That’s why I stick with a simple programming task that is rich with different programming possibilities.

When I ask a candidate to write the lowercase function , I’m deliberately vague about how to handle non-uppercase values passed to the function. I’m interested in whether non-uppercase values are considered without having to explicitly mention them. Not surprisingly, many times a candidate’s first attempt at writing the lowercase function is:

char lowercase(char c)

{

return (c + 32);

}

This code works fine if the character happens to be an uppercase letter, but the code doesn’t work if a lowercase letter or symbol is passed to the function. This is a warning flag.

Even if the candidate’s first attempt handles all types of characters, I always ask what I consider to be one of the most important questions during the interview: What values would you use to test your code?

Many candidates will answer with “capital A, B, and C...”

That response immediately tells me that the candidate doesn’t know how to test properly. A better answer to my question about testing the inputs would be “a capital letter, a lowercase letter, and a symbol.” This answer tells me that the candidate is at least thinking about the different types of possibilities. But such an answer is still lacking.

What I’m most interested in is whether the candidate will suggest test cases at the boundary conditions. Do ‘A’ and ‘Z’ work? What about the ‘@’ character which immediately precedes the ‘A’ character in the ASCII table, and the ‘[‘ character, which immediately follows the ‘Z’ character? I get concerned when a candidate can’t come up with good test cases for a function as simple as lowercase. What happens when the candidate writes a function that is more complex than lowercase?

Some programmers never come up with good test cases no matter how many hints or probing questions you ask them. Fortunately, most candidates do eventually recognize that their code will work only for uppercase letters, and when they do hit on that realization they suggest rewriting the code so that it returns the character unchanged for non-uppercase letters:

char lowercase(char c)

{

if (c >= 'A'  &&  c <= 'Z')

return (c + 32);

else

return (c);

}

More often, though, candidates will fix the problem by returning an error if the character is not an uppercase letter. They occasionally return EOF or NULL for non-uppercase letters, but for some reason, returning -1 is the most popular error value I see when conducting interviews:

char lowercase(char c)

{

if (c >= 'A'  &&  c <= 'Z')

return (c + 32);

else

return (-1);

}

I covered the -1 problem in Chapter 5 where I advised programmers that they not bury error codes in return values, so I won’t go over that again here. When I see a candidate burying error codes in return values, I always ask the candidate how he or she thinks lowercase might be used in an actual program.

After walking through different calling scenarios, I hope to see the candidate discover that returning an error may not be the best design for lowercase given how it would typically be used. I then ask the candidate if he or she can generalize the concept just discovered so they can employ it in other, less obvious, instances in the future. I’m interested in how they generalize their discovery and how they refine it with a bit of discussion.

At this point, we supposedly have a lowercase function that includes the range check for uppercase letters and that returns other characters unchanged. But frequently the range test will have an off-by-one error because the programmer used either > or < instead of >= or <=. Here, I would return to my earlier question and again ask, “What values would you use to test this code?” You’d be surprised by how many programmers answer with the same bad answer they gave before even though we’ve since talked about better test cases. That sends up another red flag.

Sometimes a programmer will use the > and < operators in the range check, testing against ‘@’, the character before ‘A’, and ‘[‘, the character after ‘Z’:

if (c > '@'  &&  c < '[')

While this may be correct, it concerns me because the code is so much harder to read than the obvious way to write the test. In these cases, I ask the candidate why they chose to write the code the way they did. Their reasoning may or may not raise a flag.

Other times, a candidate may write the test using numerical constants rather than the character constants ‘A’ and ‘Z’:

if (c >= Øx41  &&  c <= Øx5A)

When a programmer uses hard coded numbers rather than character constants in the test, I always ask them what their thought process is. It’s a great opportunity to explore whether the candidate knows the hazards of spreading hard coded constants throughout a program.

Even if the programmer writes the test in the straightforward manner, I might turn the issue around and ask why he or she didn’t use hard coded constants, or use the obscure tests we just saw. His or her answer may be informative. I use every opportunity that arises to delve deeper into their mindset, or to see how broad their practical knowledge is.

Once we have a lowercase function that produces meaningful results for an ASCII-based system, I’ll start asking questions about the code itself. If the current solution is

char lowercase(char c)

{

if (c >= 'A'  &&  c <= 'Z')

return (c + 32);

else

return (c);

}

I might ask if the else line in the code is necessary, or could the else be removed and the code be rewritten as:

char lowercase(char c)

{

if (c >= 'A'  &&  c <= 'Z')

return (c + 32);

return (c);

}

I don’t care whether the programmer uses the else clause in this particular example because the else clause will be optimized out by the complier. Still, the question can be insightful. Some programmers will tell you that it doesn’t matter if you drop the else clause. Other programmers will suggest that keeping the else clause is better programming style.

Both answers are fine with me.

The red flag goes up when a programmer says, “the else clause is required or the function will return twice” or “the function will return two values but the caller is expecting to only get one.” That’s bad. It tells me that the programmer doesn’t understand fundamental programming concepts. And for the curious: Yes, I’ve heard these responses many times in my years of conducting interviews.

Some programmers will initially write the else-less version of lowercase. In those cases, I’ll turn the else question around and ask if it would hurt to insert the else. Some programmers say, “No, the else will be optimized away,” and I ask, “What exactly will be optimized away?”

Other programmers say that adding the else would be detrimental because it would generate extra code. To them I ask, “What exactly will be generated?”

Either way, I’m curious about whether they know that an else typically generates an unconditional branch in unoptimized code.

I don’t expect graduates right out of college to know this level of detail, but I do expect programmers who have worked awhile in the field to know something about how compilers generate code. It’s a breadth of knowledge question.

Consider this: How effective can a programmer be at making size and speed tradeoffs if they have no understanding of how even the simplest of language constructs is generated by the compiler?

The question of whether to use or drop else in this example brings up a point about programming style. Sometimes a programmer will studiously avoid using multiple returns when writing the lowercase function because they were taught that it’s bad style. That’s fine with me—it’s not my goal in an interview to play style-cop. My job is to determine the programmer’s skill level.

I will, however, ask the programmer why he or she feels that it’s bad style to use multiple returns. If it’s purely for “religious” reasons, I’ll take a brief detour to touch on other “religious” programming topics such as brace placement, tab-widths, using gotos, naming conventions, and so on. I get concerned if a programmer is too inflexible, particularly when he or she can’t justify his or her position—they just know that it’s right, or wrong, or whatever. Inflexible thinking raises a red flag because it’s often more difficult to work with such people. By itself, this wouldn’t prevent me from hiring someone, but if there were other flags raised in the interview, it could push me the other way.

To get a feel for how the candidate would attack a real-world issue, I might ask how she could make the function faster. Her reply might be, “We could get rid of one of the returns.”

char lowercase(char c)

{

if (c >= 'A'  &&  c <= 'Z')

c += 32;

return (c);

}

My next question would then be, “How do you know this new version is faster?” If she says she would compare the two versions with a code timer, or compare the code generated by the compiler, adding up instruction cycles, I’d accept that—she clearly understands how to verify her proposed improvement.

What I don’t like to hear is “It’s obvious—I’ve eliminated duplicate code.”

I don’t know about you, but it’s not obvious to me that the new code would be faster given the quality of today’s optimizing compilers. The change is too small. Skilled programmers always verify their assumptions.

I should mention again here that with the blinding speed of modern processors, such miniscule differences in code speed don’t matter the way it did decades ago when every processor cycle saved in your design was critically important to performance. That said, programmers still need to understand these concepts because when developing subsystems, this knowledge is still relevant. If you’re developing an SQL database, a memory manager, or a file system, you need to have a grasp on these programming tradeoffs.

At this point I might put a spin on the speed question, asking the candidate if she could make lowercase any faster if she knew that lowercase was being passed lowercase letters 87% of the time.

Less experienced programmers will peruse their code for a while and eventually conclude that there’s nothing they can do—there’s no way to avoid the range check.

More experienced programmers quickly rewrite the test, often using short-circuit evaluation found in many languages:

char lowercase(char c)

{

if (c > 'Z'  ||  c < 'A')

return (c);

return (c + 32);

}

Programmers who aren’t familiar with short-circuit evaluation should still be able to come up with a back-to-back if statement solution:

char lowercase(char c)

{

if (c > 'Z')      /*  Quick exit for lowercase letters  */

return (c);

if (c < 'A')

return (c);

return (c + 32);

}

Because handling the common case quickly is the fundamental concept behind performance tuning, you can be reasonably sure that programmers who have trouble handling this optimization task have never done any type of performance tuning. For the same reason, you can be reasonably sure that such programmers haven’t done much program design. This is another case where, if necessary, I’ll walk the programmer through the problem until he or she discovers the concept for himself or herself. Then to cement the concept, I’ll ask the candidate to generalize it for me. The interview might as well be educational, particularly since the candidate could soon be an employee.

I always ask this next question during the interview: Suppose you had gobs of memory. Would that help in writing a faster version of lowercase? I believe programmers should consider table solutions whenever memory is not much of a concern. With this question, I hope to see the programmer come up with at least a conceptual table-based lowercase:

char ascii_lowercase[] = { table initializers };

#define lowercase(ch) ascii_lowercase[(unsigned char)(ch)]

Programmers should also be able to give an intelligent discussion of the pros and cons of table-lookup implementations. Pros: they can be much faster than algorithmic functions and the code to implement them is typically short, robust, and easy to understand. Cons: lookup tables can be big, and the larger the table, the greater the chance for erroneous table entries particularly if the contents are not machine-generated.

During the interview, I keep asking about different design choices and delve into their thinking behind their answers. I’m looking for their underlying philosophies about writing code. Do they have stinkin’ thinkin’ or do they hold good, solid, philosophies about how to write fast, clean, and rock-solid code?

With such an interview, I can develop a good feel for a programmer’s skill level, including the candidate’s design skills even though the interview doesn’t touch much on major design issues.

A mentor of mine always says, “How you do anything, determines how you do everything.” If a programmer can’t write a useable tolower function, they don’t have a prayer of implementing something more substantial.

If you’re in the position to determine who gets hired, I’d encourage you to at least try doing this type of interview. I use tolower for entry-level positions, but if I’m hiring for a more advanced position, I use a small easy-to-understand task that tests for the skill level I need.

REFERENCES

These books are explicitly referenced in the text.

American National Standards Institute (ANSI). Programming Language-C. New York: American National Standards Institute, 1990.

Cialdini, Robert B. Influence: How and Why People Agree To Things. New York: Morrow, 1984.

Kernighan, Brian W., and P.J. Plauger. The Elements of Programming Style. Second Edition. New York: McGraw Hill Book Company, 1978

Kernighan, Brian W., and Dennis M Ritchie. The C Programming Language. Englewood Cliffs, N.J.: Prentice-Hall, 1978. Second Edition 1988.

Knuth, Donald E. TEX: The Program. Menlo Park, CA: Addison Wesley, 1986; reprinted with corrections 1988.

Lynch, Peter with John Rothchild. One Up On Wall Street. New York: Penguin Books, 1990.

Motorola. M68000 16/32-Bit Microprocessor Programmer’s Reference Manual. 4th ed. Englewood Cliffs, NJ: Prentice-Hall, Inc., 1979.

Plauger, P.J., and Jim Brodie. Standard C. Redmond, WA: Microsoft Press, 1989.

Plauger, P.J. The Standard C Library. Englewood Cliffs, NJ: Prentice-Hall, Inc., 1992.

Robbins, Anthony. Awaken The Giant Within. New York: Summit Books, 1991.

Simonyi, Charles. “Meta-Programming: A Software Production Method.” Thesis, Xerox Palo Alto Research Center, 1977. Also issued as CSL Report 76-7 Xerox Palo Alto Research Center, December 1976.

ABOUT THE AUTHOR

Although Steve Maguire has a degree in Electrical and Computer Engineering from the University of Arizona, he has always been more interested in the software end of the computer business. He has been programming professionally since 1978 and has worked in Japan as well as the United States.

In the late 1970s Steve was active in the microcomputer arena and was a regular contributor to the Processor Technology and NorthStar users' groups, contributing programmer tools, applications utilities, and the occasional video game. Since then, Steve has been responsible for numerous projects, including valFORTH in 1982, an award-winning FORTH development system that allowed Atari programmers to write high-quality graphics applications and video games.

In 1986, Steve accepted a position with Microsoft Corporation specifically to work on high-end Macintosh applications. In addition to working on Microsoft Excel v1.x, Steve led the development of Microsoft's Macintosh cross-development system. He was the driving force behind Microsoft's switch to a cross-platform, shared code strategy in its applications development and is perhaps best known in the company for his efforts to increase the utility and quality of shared code libraries.

His first book, the critically acclaimed Writing Solid Code (Microsoft Press, 1993), focused on strategies that programmers can use to write bug-free programs. It won a prestigious Software Development Jolt Productivity award and awards from the Society for Technical Communication in 1994. The second edition of Writing Solid Code builds on that earlier success.

Steve’s second book, Debugging the Development Process, was written to give programmers practical guidelines for developing professional, high-quality software. It also won the prestigious Software Development Jolt Productivity award.

Today, Steve is a Senior Level developer and Project Lead for Storm Development, a web development company based in Cincinnati, Ohio. When Steve is not writing code, you can find him walking his dogs in the woods with his wife or out sailing on a nearby lake.

You can contact Steve by emailing him at steve@stormdev.com. For more information visit the book’s website at www.WritingSolidCode.com.

COPYRIGHT

Writing Solid Code

Copyright © 2013 by Stephen A Maguire

Storm Development, Cincinnati, OH 45242

www.WritingSolidCode.com

All rights reserved. No part of the contents of this book may be reproduced or transmitted in any form or by any means without the written permission of the publisher.

TEX is a registered trademark of the American Mathematical Society.

Apple and Macintosh are registered trademarks of Apple, Inc.

MS-DOS, and Windows are trademarks of Microsoft Corporation.

COLOPHON

Printing History: First Edition 1993 by Microsoft Press

Acquisitions Editor: Mike Halvorson

Project Editor: Erin O’Connor

Technical Reviewer: Wm. Jeff Carey

Principal editorial compositor: Cheryl Whiteside

Principal proofreader/copy editor: Kathleen Atkins

Principal typographers: Jean Trenary and Carolyn Davids

Interior designer: Kim Eggleston

Principal illustrator: Lisa Sandburg

Cover designer: Rebecca Geisler

Second Edition:

Project Lead for Second Edition:

Dave Braughler, Braughler Books 2013

Cover Photographer:

Paul Cooklin / Brand X Pictures / JupiterImages

eBook:

Torch Lake Press / Kewadin MI, USA: 2021

The manuscript for this book was prepared using Microsoft Word for Macintosh on a 27”Apple iMac running macOS Catalina.

The body text for the printed version of this composition uses Palatino Linotype 10. Chapter titles use Palatino Linotype 48, italic, with small caps. Section headings use Palatino Linotype 16, with small caps. Code examples use Lucida Sans Typewriter 8.

Table of Contents

Contents

Foreword

Preface

Acknowledgments

Introduction

Two Critical Questions

What Are Those Gobbledygook Names?

Just What is a Bug?

Automatic Bug Detection

Watch Your Language

lint—It’s Not That Bad

But My Changes Were Trivial

No More Egg Rolls

Assert Yourself

A Tale of Two Versions

Introducing Assert

“Undefined” Means “Steer Clear”

The Code That Cried “Danger”

Not For Error-Handling

Are You Making Assumptions Again?

Can the Impossible Happen?

The Silent Treatment

Two Algorithms Are Better Than One

Stop Bugs at the Starting Line

Assertions Are Forever

Fortify Your Subsystems

Subsystem Checks

Shred Your Garbage

Keep A Journal to Jog Your Memory

Don’t Wait for Bugs to Call

It’s Obvious Once You See It

There’s No Need To Know

Not All Designs Are Equal

A Paradigm Shift

Find Bugs Now, or Find Them Later

Step Through Your Code

Gain Confidence in Your Code

Forcing Errors in the Code

Data Flow, the Lifeblood of Code

Are You Missing Something?

Try It—You’ll Like It

Candy-Machine Interfaces

getchar Gets An int, Of Course

Just a Little Extra Thought

The One-Function Memory Manager

Wishy-Washy Inputs

Don’t Fail Me Now

Read Between the Lines

Warn People of the Hazards

The Devil Is in the Details

Risky Business

Doth Your Data Run Over?

Close Counts Only in Horseshoes

Functions Just “Doing Their Thing”

No ifs, &&s, or Buts

The ?: operator is an if statement too

Rid Your Code of Redundancy

High Risk, No Return

Inconsistency, the Gremlin of Code

Don’t Associate with Failures

A Slap on the Risk

Treacheries of the Trade

The Need for Speed

A Thief With a Key Is Still a Thief

Take Only What You Need

Keep Private Things to Yourself

Functional leeches

The Old Paint-Stirrer Trick

APL Syndrome

No Hoity-Toity Programming

Waste That Bag of Tricks

The Rest is Attitude

For my next trick, disappearing bugs

A Fix in Time Saves Nine

Bug-Doctor to the Rescue!

Are You A Code Meddler?

Put “Cool” Features Into Cold Storage

No Free Lunches

Flexibility Breeds Bugs

Ported Code is New Code

“Try” Is A Four-Letter Word

The Sacred Schedule

What’s in a Name?

Testers Wear White Hats

What Are Your Coding Priorities?

You Don’t Get What You Don’t Ask For

Where Do You Go From Here?

Coding Checklists

Design

Implementation

Adding Debug Support

Testing

Debugging

Memory Logging Routines

Answers

The Interview Question

References

About the Author

Copyright

Colophon