## Friday, March 30, 2007

### Unsafe String Concatenation in C

Many remote exploit vulnerability can be classified into two kinds: memory management problems and confusion of language domains. Memory issues are characteristic of programs written in a low-level language like C. Confusion of language domain is more characteristic of programs written in a high-level scripting language like Perl, PHP and Python. For networked applications that require performance, programmers tend to use a low-level language like C, so memory issues like buffer overrun has been a primary concern in this case.

Because buffer overrun has been Numero Uno cause of security issues, courses and programming books teaching C nowadays have (hopefully) delivered good practices of memory management. One example is to use string functions that limit buffer size. For example, use snprintf() instead of sprintf(), fgets() instead of gets(), strncpy() instead of strcpy(). These functions provide an argument to impose buffer usage limit in order to avoid overrun.

One may be inclined to also recommend strncat() in place of strncat() for string concatenation. However, this deserves closer scrutiny. This function has the following prototype:
char *
strncat(char * restrict s, const char * restrict append, size_t count);
It copies at most count characters from append to the end of s. However, if s is nearly filling up its buffer size, a count that is too large would still overflow the buffer that holds s. We can illustrate this using the following example:
char buf[16];
strcpy(buf, "hello world!");
strncat(buf, " adam!", sizeof(buf)); // wrong!
This code instructs strncat() to append at most 16 bytes, which is sizeof(buf), from " adam!" string. Obviously, strncat() ends up appending the whole string, and this overflows buffer by 3 characters because the string "hello world! adam!" is 18 characters in length plus the terminating NUL.

This is apparently a real problem that once plagued CUPS (printing system for Linux and Mac OS X) and Solaris ufsrestore (incremental file system restore).

Another easy to make mistake is the fact that strncpy() may not properly zero-terminate a string. For example:
char buf[4];
strncpy(buf, "love", sizeof(buf));
printf("buffer contains the string '%s'", buf);
The code would end up printing "buffer contains the string 'love...'" with some garbage. The reason for the garbage is that buffer is no longer properly NUL terminated. You're lucky if the garbage is relatively short. Trying to pass buf to a string processing function that expects NUL terminated string may cause that function to run forever, even for benign functions like strlen().

On the other hand, fgets() reads a string into buffer up to size - 1 characters and properly NUL terminates the string.

All in all, I think these C functions are examples of bad interface design. If someone is reworking the string functions in C, he should focus on making the interface consistent, namely:
1. That strncat function, like everyone else, should accept destination buffer size limit, rather than limiting number of characters to append; and
2. All string functions should result in valid, properly NUL terminated strings.
Furthermore, I actually encourage one to use strcat() and strcpy(), despite them being allegedly "unsafe." The rationale is that this forces you into the habit of always checking string length with strlen() and make sure you have enough buffer size to hold the resulting string before performing the operation.

There are additional gripes about strtok()/strsep()—they destroy the original string, which is a problem if the string is shared with another data structure or function. You should use strchr() or strcspn() to find the next occurrence of delimiter, and use strncat() to copy the delimited portion to an empty—but NUL terminated—string. The reason why you shouldn't use strncpy() is precisely that it does not NUL terminate your string.

You should consider strncat() as a safer substitute for strncpy() because strncat() at least guarantees NUL terminating the resulting string (string copying is a special case for concatenation). This also brings up an issue about how to initialize a string buffer. Instead of zeroing out every bytes in the buffer, you only need to zero the first byte.

We discussed a number of string processing functions and their interface irregularities, and we showed some workarounds for better security practice. This illustrates the amount of detailed attention one has to pay when writing C programs. I hope that students who first learn C would just pick up the good practice from the beginning.

## Wednesday, March 28, 2007

### PDF editing on Linux (and other open source OS)

I recently received several PDF documents to fill out for background search (I'm applying for summer internship). However, the PDFs did not use proper form fields, so I cannot use Acrobat Reader to fill them. I surveyed two PDF editors that serve this particular purpose—add text to an existing PDF without form.

Before I began, I did some search on Google. I found this article (mentioning flpsed) that looks somewhat outdated. I also remember reading about a PDF editor based on QT (PDFedit) a while ago. I decided to try both.

I tried PDFedit first because it looks more recent. According to the user documentation, it is quite powerful. It has a Javascript-like scripting language (QSA) that let you automate many things. It can alter existing text attributes, delete objects, add lines and rectangles, and add text. It also lets you view PDF object tree and edit it directly. PDFedit supports multi-page documents, which Adobe Illustrator doesn't.

As for the function I was looking for, adding text to a document to fill out forms, PDFedit performs rather poorly. The button to add text does not have an obvious icon. Once you find the button and click on it, your mouse pointer turns into "add text" mode, so you can now click anywhere in the document to add text.

When you click to add text, PDFedit creates a small overlaid text box that lets you type. However, if you type too much, parts of your typing is scrolled away and becomes temporarily hidden. When you press the Enter key, everything you just typed (including the occluded part) is rendered onto the document, but with a different font and size. The desired font can be chosen using a drop-down box on the toolbar, but this setting is not previewed when you type.

Once you entered the text, you may discover that the placement is slightly off. You must switch the mouse back to selection mode, select the text you just entered, and drag it. Mouse selection does not work well, and you can easily select the wrong object or some mysterious object not visible in the document.

After adding text, dragging, and adding more text for a few times, PDFedit becomes very slow. On a fairly recent machine (Pentium 4, 2.4Ghz), it can take 20 seconds from pressing the Enter key until text appears in the document.

When I finally struggled through, I saved the resulting PDF file and tried to open it in Acrobat Reader. It opened the document only partially and complained about a "q" operator that is illegal in text. I tried opening in GhostView, and it just rejected the document completely. I tried loading it in PDFedit again, and it was fine. It seems that after you tinker a PDF with PDFedit, the only program you can ever open it again is PDFedit.

I decided to give flpsed a try. The reason I didn't try it first is because the article I read claimed that it only supports PostScript files. It wasn't that big of a problem because PostScript can be converted back and forth from PDF using GhostScript, but with some loss in font details.

Compiling flpsed from source was a pleasure. It requires fltk 2.0, which took only a minute to compile; flpsed itself took only a few seconds. On the contrary, PDFedit requires some functionality from boost, and both took me tens of minutes to compile.

With flpsed, I first tried converting PDF to PostScript, and it worked fine. The program is simple and serves only one purpose: add text to an existing document. It also supports multi-page documents. I can choose font sizes, but not font family. Adding text is simple—you click on anywhere in the document and start typing away. You can only edit what you typed, but you cannot modify original document text.

Flpsed is also much more responsive than PDFedit. Typing, selecting, and moving text is instantaneous. You can also move text with your keyboard cursor keys for more precise alignment.

After finishing the forms, I discovered that flpsed also imports directly from PDF and exports back to PDF. This actually produces better result than doing PDF to PostScript conversion. I started over with PDF, and it was a breeze to do.

However, the simplicity of flpsed comes with functionality trade-off. For example, there is no cut and paste function. It also doesn't have input method support (which is required for many non-English languages). I also cannot add image—this is to simulate signing a document, although it may be a bad idea to allow your signature to be infinitely reproducible by putting it inside a printable PDF document.

Here is my recommendation: if you need to fill PDF "forms" that do not have form fields, flpsed does the job beautifully and efficiently. But don't expect it to do much else.

## Wednesday, March 21, 2007

### Micro-blog Idea

A much desired function of my calendar is the counter-part of the TODO list, that is, the DONE list. I need to keep a list of things I have done on a day to day basis, even if it was not on TODO before. It will also serve as a memo taker---some sort of a micro-blog, if you will. It will be different than a TODO list, where you cross things out when something is done and you forget about it. I want to remember what I have done and go back to it when I need to.

I need it because that's the only way I could show my professor that I have done reasonable work whenever he throws at me the "Angus, you're not sticking to your agenda" crap.

This micro-blog would be split up in different streams, some could become public, and others would remain private. It would be browsed over the web or pulled in as an RSS/Atom feed.

Now, it strikes me that SQL database is probably not the best way to implement the back-end for any sizable web application. None of the SQL servers I know of can distribute both load and storage across different nodes the way Google File System does it. SQL, the language, abstracts very little from bare metal, and many higher-level languages like ML that support higher-order functions can do much better in terms of abstraction.

Remember MapReduce from Google? The concept is so prevalent in functional languages that one can almost accuse them of reinventing the wheel. With more advanced compilers (like what Haskell has) that perform deforestation, you can even fuse together a number of MapReduce steps so you do not need to generate intermediate data set. Deforestation allows you to write easy to understand, pipe-lined programs without compromising efficiency.

I think there are many ways that databases can benefit from advanced features of a programming language. For example, the list that results from traversal of a tree index can be implemented as a lazy-evaluated stream, where the resulting list is generated on-demand. Many stream based operations are readily available, such as fold, map, and filter. Other fancy stream operations are also possible, such as taking the cross-product, distribute a set of lists in a round-robin fashion, and doing merge sort.

However, a major short-coming of any functional language today is the lack of a good XML library. People in this community have designed plenty of esoteric, experimental XML parsers and generators, but they are difficult to use in practice. It needs to be as easy to use as Javascript to DOM without sacrificing the power of functional programming, e.g. pattern matching, fold, map, and combinators---I don't know, maybe these are mutually conflicting goals.

We also lack a good HTTP client library. Ocamlnet is very powerful, but it is way too involved for casual usage. I think XMLHttpRequest is a great API that someone should implement. Its greatness is evident from the success of Web 2.0, also known as AJAX.

I'm afraid I have to leave this post inconclusive. I've drifted from an idea to the critique of possible solutions. In the end, I think the saying that ideas are worth (...insert your favorite object of no value here...) is true. I can only make a difference if I actually implement it.

So little time, so much to do.

DONE: spent 2 hours in the morning writing about a worthless idea.

## Monday, March 19, 2007

Computer systems have become so complicated that it is likely to misdiagnose the problem if you look only at the symptoms and lack understanding of how it works.

A friend's old Windows XP computer was getting the error "Windows---Display Driver Stopped Responding: The i81xdnt5 display driver has stopped working normally," and he asked me to look at it. Whenever it happens, the display switches back to 640x480 resolution at 16 colors and messes up desktop icon arrangement. He could restart the computer, but it gets annoying.

A quick search turns up a Microsoft Knowledge Base article explaining that the device driver might be faulty and suggested that we upgrade the display driver. We tried that but it didn't help.

He said he has a way to reproduce it---the error happens when he runs a lot of applications. I suspect it has something to do with virtual memory issues, so I checked:
1. If the disk is running low on available space. But it had plenty, with 9GB free.
2. How much memory is free, using Task Manager. I discovered that Windows only uses up to 384MB of page file, and it is nearly full right after the system was rebooted.
3. The performance setting, and found out that someone had limited page file size to 384MB.
I changed the page file size to "system managed" and solved the problem.

The computer originally had only 128MB of ram, but I helped my friend upgrade to 256MB at some point. I think whoever bought the computer for my friend before had followed the rule that page file should be three times the physical memory size, but the memory upgrade requires the size limit to be updated. That's why he's running into this problem only after memory upgrade.

I can see there are many possible misdiagnoses:
1. As the Microsoft Knowledge Base states, it could be due to a faulty display driver. Cost of misdiagnosis is the time to find display driver and install it; not bad so far.
2. It could be due to a faulty hard drive for failing to provide a reliable page file. We had a hard drive problem on that computer before. It was fixed after a chkdsk, but that could indicate that the drive is dying. Cost of misdiagnosis is ~$80 for a new hard drive. 3. Since the problem only happened after a memory upgrade, it could be due to the new memory modules. Cost of misdiagnosis is ~$100 for a new memory module.
4. It could be due to a faulty display card for "not responding" to a driver. However, the computer uses integrated graphics chip on the motherboard, so that could imply the motherboard needs replacement. Cost of misdiagnosis is either ~$50 for a new display card, or ~$150 for a new motherboard; trying both and finding out that neither help cost ~$200. As you can see, it could easily cost us$300+ sans labor if we had tried everything we can imagine under the sun.

Think about the implication of a medical misdiagnosis. Human bodies are much more complicated than computers.

I am having difficulty concluding the moral of a story I'm about to tell you, so please bear with me while I tell the story first.

I've recently started performing an upgrade of sshd on a multi-user server from original ssh 2.0.13, circa 1999, to the latest OpenSSH, 4.6p1 at the time of writing. The reason is complicated. The server is a Solaris 2.6 system that has been in use since before 1999, and it is already falling behind on security patches. The ssh is very old---protocol 2 had some bugs, and we always have to fall back using ssh protocol 1 instead, which has long been deprecated.

Several people and I have root access to that server, but nobody assumed the position of a system admin. I think that's a case of collaborative irresponsibility. The department has an appointed system admin who is supposed to do the job, but he has been reluctant in keeping that server up to date especially after the department's migration to Linux a few years ago.

Some time ago, I read about yet another telnet remote root exploit and decided to disable all insecure means of login to that particular server, which are telnet, rsh, rlogin, rexec, and ftp. About a month later, I got an e-mail from IT backup service telling me that backup server cannot login using rsh. My professor asserted that I should find a resolution because I was the one who disabled the insecure services. Tell me about how it feels to be rewarded with mounting responsibility for a moment of attentiveness.

I told IT to use ssh, but they insisted on using a ssh protocol 2 key, and that was the last straw that prompted me to take the initiative to upgrade sshd.

After installing the software and converted the host keys, I thought it would be nice to run the new sshd on a separate port for testing. It allowed other people to help me iron out a number of problems, like the infamous "Disconnecting: Corrupted check bytes on input" problem when using ssh protocol 1 from an older client (the solution is to make cipher-3des1.c and cipher-bf1.c include openssl-compat.h).

However, the very evening after I ran sshd for testing, the department system admin was alerted of a possible intrusion. Although I already told IT that I would upgrade sshd, another branch of IT---the intrusion response team---didn't know about it and thought it to be suspicious that sshd is being run on another port. This resulted in the server being taken offline for a few hours until I had a chance to intervene.

Well, what's the big deal? The original idea of running a testing sshd on a separate port is to minimize downtime, so that our group members could continue using the old sshd while I prepare the new sshd for production. It ended up causing more downtime. The measure backfired because of the false intrusion alarm.

How do they tell there is another ssh service running on suspicious port? You can just scan all open ports and see which one responds with the string "SSH-1.99-OpenSSH_4.5" that is not from port 22. I think this is pretty absurd though; I could play a prank by ssh into a legitimate host and use port forwarding to tunnel its own ssh port to any other port I want on this host, e.g. ssh hostname -R10022:hostname:22 -R11022:hostname:22 -R12022:hostname:22 and so on. This will make the system admin sweat without me violating any computing policies.

I think there are a number of morals of this story:
1. It is always a good idea to let others help you whenever you can. I knew about the "corrupted check bytes on input" problem, but another person helped me identify the fix because I invited everyone to test the new sshd. There are other times when I could have helped the department system admin if he were more responsive to suggestions and problem reports.
2. In an organization where one hand doesn't wash another, even a thoughtful plan can run into problems. In this case, a well-intended test bed triggered false intrusion alarm. These problems aren't always fatal, but they require overhead to fix. Optimistically speaking though, I think experience can help someone avoid such problems.
3. Don't be so keen on taking care of something unless you want the responsibility to be assumed upon you. Alternatively, I think I could have done a better job promoting to my professor the value of the additional "not my jurisdiction" work that I do. Managing your superior is a difficult art to master, but that can be necessary.
Nevertheless, the upgrade is now completed, and several people are satisfied.