MrHacks.tumblr.com

Why Python doesn’t do printf by default

I kinda left out an important part to yeterday’s article that I really wanted to write about, but my habit of going off into tangents sometimes when I write kinda made me forget the important part. So I will be brief today with my wandering explainations.

They way alot of people teach C++ is of ill justice to the student who learns about programming if the student has never programmed in C. Sure, C++ has all sorts of great features that C doesn’t have. But if anyone is going to teach a student C++, they should teach them how to use C or how program C-style in C++.

Case in point, the much revered printf function. (I’d like to discuss scanf as well, but will reserve that for another time in the near future.) PHP, Perl, and Bash support printf, but in Python, print is the sheriff in these parts.

I already stated how Python uses print.

#!/usr/bin/python
# File: meat.py
red = "\x1b[1;31m"      # set the string color to red
grn = "\x1b[1;32m"      # green
blu = "\x1b[1;34m"      # blue
rst = "\x1b[0m"         # reset the string back to normal (required!)

print "%sHello %sColorful %sWorld%s" % (red,grn,blu,rst)

But just for fun, what if we wanted to use the C function printf? Let’s start out with a simple C program. (For even more kicks, I’ll write it as a C++ program later in this article.)

/* File: carne.c
 * Info: The C-version of carne.py
 */
#include 
 
typedef const char* cpstr;

int main(){
 /* string colors */
 cpstr red = "\x1b[1;31m";      /* red */
 cpstr grn = "\x1b[1;32m";      /* green */
 cpstr blu = "\x1b[1;34m";      /* blue */
 cpstr rst = "\x1b[0m";         /* required reset */
 /* printing a string */
 printf("%sHello %sColorful %sWorld%s\n",red,grn,blu,rst);
 return 0;
};

Note that unlike in Python, C and C++ programs must be compiled and sometimes linked. Fortunately, this program doesn’t need anything fancy to do the the compiling, and because it is one file, linking is not necessary. So we can do the following steps in the shell.

$ gcc carne.c -o carne
$ ./carne
Hello Colorful World

No different from meat.py, right? So let’s have some fun with this program and convert it from a C file to a C++ file using sed. (Here be a little hacker exercise for a very simple program. Normally, I do my edits in vim, but I’m feeling a bit techie this morning.

It is important to note, that C++ uses namespace. While some people go the route of adding the line using namespace std;, this is reckless, not to mention one of the annoyances of C++. Needless to say, namespaces are still useful. I just don’t have a complete knowledge of them to use them so lets move on.

C++ also has special names for many of the standard C header files. Often, anything from the C library that is used in a C++ program will likely need a std:: inserted before it. Of course, this may vary in some places.

There are two ways that I could probably use sed to make changes. The first way is that I could use sed interactively. Though before I do that, I would want to make a copy of carne.c first. This way is ideal for Linux newbies and for folks that want to edit files quickly or compose a list without opening a text editor. (I’ll explain how to use sed for making lists probably in the near future but in a different post.) The second way is to use a streamed chain of commands, which is more ideal for expert programmers or for use inside a bash script. I’ll show that later.

First, let’s start by making a copy of the original file.

$ cp carne.c carne_sed.cpp

Before I continue explain how to use sed to convert this file, there are a few ways that I can see the contents inside a file without going into a text editor. The best know way to do this is using cat. But for long files, not everything can fit on the screen.

$ cat carne.c
/* File: carne.c
 * Info: The C-version of carne.py
 */
#include 
 
typedef const char* cpstr;

int main(){
 /* string colors */
 cpstr red = "\x1b[1;31m";      /* red */
 cpstr grn = "\x1b[1;32m";      /* green */
 cpstr blu = "\x1b[1;34m";      /* blue */
 cpstr rst = "\x1b[0m";         /* required reset */
 /* printing a string */
 printf("%sHello %sColorful %sWorld%s\n",red,grn,blu,rst);
 return 0;
};
$

Terminal programs like GNOME Terminal and KTerminal has scrollbar, but they only go back to a certain number of lines, and you shouldn’t use it at all if you are running a text editor in the terminal. So, if you are a Linux user, you must train yourself to swear off the scroll bar unless you are writing a program with a GUI, to which I won’t explain how to do that in this blog. Maybe in OpenGL or PyTK, but for now, if it ain’t in the terminal, it’s not discussed here.

So if by chance you find yourself in a situation where there are no scrollbars, your best bet is to pipe data into the less command.

cat carne.c | less -eFMXR

The -eFMXR part is not necessary when using less. I just like to add it because it offers a few extra options in case I needed them. Here’s the low down on what each of these attributes does, though a more through explation can be found in the man pages (man less).

• -e quits less when I reach the EOF (end-of-file) twice—that is, I would need to press the down arrow twice at the end before quiting.
• -F quits less if everything fits on one screen.
• -M is really optional, especially since it’s kinda useless when used with piped or streamed input. If It would let me it would show my position in the file, but now that I think of it, I might just not use this in the future. It’s still worth mentioning though.
• -R is very important. -R processes raw control characters, which is important if want to see color text output when using ls or grep. Both of those commands have a --color=always attribute which must be used with less -R. However, -R may still run into uses like if CTRL-v + ESC + [ are used. It is for this reason, I use and encourage the use of the \x1b[ sequence in these examples to avoid that issue.
• -X tells the terminal to use termcap initalization and deinitialization. You may have seen this when using vim, man, or info. Not using termcap is a must if you are writing bash scripts that stream output to the shell.

Of course, even with these options, you can quit less by typing the letter q where ever.

I wanted to talk about using grep, but that would divert us futher away from the subject of using sed, of which there is a sed command that for the most part does something that grep does but without the color. Printing lines.

sed’s line printing hack is usefull when you want to pull up data from specific rows. gawk and cut can do the same thing with columns, which is why learning how to use a text-file as a textual database before diving in to something like SQL is not only idea, but very easy to do…AND you can find data in files that probably should have been processed as such. Again, another topic for another time.

Let’s start out with this simple little example.

$ sed -n "/cpstr/p" carne.c
typedef const char* cpstr;
 cpstr red = "\x1b[1;31m";	/* red */
 cpstr grn = "\x1b[1;32m";	/* green */
 cpstr blu = "\x1b[1;34m";	/* blue */
 cpstr rst = "\x1b[0m";		/* required reset */

Even the spaces in the indentation appear. Of course, grep would have highlighted each of the terms.

$ cat carne.c | grep "cpstr"
typedef const char* cpstr;
 cpstr red = "\x1b[1;31m";	/* red */
 cpstr grn = "\x1b[1;32m";	/* green */
 cpstr blu = "\x1b[1;34m";	/* blue */
 cpstr rst = "\x1b[0m";		/* required reset */

There are a few things to point out with the sed command: -n is “quiet mode”—that is it will only output the lines that the sed command apply to, namely every line that contains the pattern cpstr. (For the most part, -n is used for the p command. You probably won’t need to use this elsewhere.) The sed command use here uses regular expression (or “regex” or “R.E.” for short), a method of string matching that abides by a set of rules. I’d love to elaborate on how computer scientists camp up with them and study them, but there isn’t anything special in this RE that requires me to dwell on and discuss right now. Besides, I’m boring you enough as it is. ;-)

The RE is anywhere in the sed command between the two forward slashes. Alternatively, I could have used a list of line numbers to tell sed to pull up lines 6 and 10 through 13, but finding line numbers is tedious. Which if anything, experimenting with the sed line printing hack is a good way to learn how to use text processing programs. Should you ever find yourself learning about SQL, you will likely learn that the first thing you learn in that class is how to pull up table information use SELECT commands, which I’ve pretty much shown you thus far the same process as they are applied to source codes and text files.

But now comes the sweet part. SUBSTITUTIONS! These are probably the most important commands used in any text processing program.

When using commands that change a file, the file will NOT be modified by any of sed comands, unless you use the -i attribute. If you don’t use that, even subsitutions are safe to use. So ALWAYS test your sed commands before you apply them.

Enough of this jibba-jabba. Make with da code, foo!

First we make our test changes. You will notice that everytime you make a subsitution, the output appears. This is incase you wanted to redirect the differences to another file, like this.

$ sed "s/C-version/C++ version/" carne.c > carne.cpp

But for now let’s stick with testing things out first. We know we want to change the file and the file information in the comments. To do that, we pipe two sed command using the following line.

$ sed "1s/carne.c/carne2.cpp/" carne.c | sed "2s/C-version/C++ version/" -
/* File: carne2.cpp
 * Info: The C++ version of carne.py
 */
#include 
 
typedef const char* cpstr;

int main(){
 /* string colors */
 cpstr red = "\x1b[1;31m";	/* red */
 cpstr grn = "\x1b[1;32m";	/* green */
 cpstr blu = "\x1b[1;34m";	/* blue */
 cpstr rst = "\x1b[0m";		/* required reset */
 /* printing a string */
 printf("%sHello %sColorful %sWorld%s\n",red,grn,blu,rst);
 return 0;
};

I know some of you are thinking there might be a different way to do this. But for now, I’ll use this method to explain it easily. Take note that neither sed commnd has the -i attribute, therefore carne.c has not been modified. The pipe (|) tells the first second sed command to use the first sed commands data as input. In many cases, the pipe is all you need to tell a sequence of commands that are working together to pass data from one command to the other. However, sed happens to be one of those program that wants a file argument when processed. That is where the hyphen (-) at the end of the command becomes necessary. Also, did you notice I told the s command where to look? Lines 1 and 2 respectively. This next set of examples might require require telling s where to make specific substitutions.

Let’s try something out first. Say I wanted to swap cpstr for const char* on lines that contained the string red. Here is why we must take care and be specific when we tell sed to apply thing in certain places. For your convenience, I will highlight what went right and what went wrong and why.

$ sed "/red/s/cpstr/const char\*/" carne.c 
/* File: carne.c
 * Info: The C-version of carne.py
 */
#include 
 
typedef const char* cpstr;

int main(){
 /* string colors */
 const char* red = "\x1b[1;31m";	/* red */
 cpstr grn = "\x1b[1;32m";	/* green */
 cpstr blu = "\x1b[1;34m";	/* blue */
 const char* rst = "\x1b[0m";		/* required reset */
 /* printing a string */
 printf("%sHello %sColorful %sWorld%s\n",red,grn,blu,rst);
 return 0;
};

Here is the precarious part about regular expressions. In some programs, they will pick up parts of words, which is why the rst’s data type was changed from cpstr to const char*. The program would have still worked. After all, typedef are really used for creating synonyms for other data types. But what if this didn’t happen like that, like if the variable was named tcpstr? That would have messed things up! A good resolution to this problem, when testing your lines, is to use the line printing hack, which s can work with p.

$ sed -n "/red/s/cpstr/const char\*/p" carne.c 
 const char* red = "\x1b[1;31m";	/* red */
 const char* rst = "\x1b[0m";		/* required reset */

You’ll notice the same error, but look what happens when I be more specific about which red to change by using / red/ (with a space) instead of /red/

$ sed -n "/ red/s/cpstr/const char\*/p" carne.c 
 const char* red = "\x1b[1;31m";	/* red */

We only made one change this time and in the right spot. Removing -n and p from the line would show how it should be.

$ sed "/ red/s/cpstr/const char\*/" carne.c 
/* File: carne.c
 * Info: The C-version of carne.py
 */
#include 
 
typedef const char* cpstr;

int main(){
 /* string colors */
 const char* red = "\x1b[1;31m";	/* red */
 cpstr grn = "\x1b[1;32m";	/* green */
 cpstr blu = "\x1b[1;34m";	/* blue */
 cpstr rst = "\x1b[0m";		/* required reset */
 /* printing a string */
 printf("%sHello %sColorful %sWorld%s\n",red,grn,blu,rst);
 return 0;
};

Now, back to making the final two changes before this C program becomes a C++ program.

$ sed "1s/carne.c/carne2.cpp/" carne.c | sed "2s/C-version/C++ version/" - | sed "/include/s/stdio.h/cstdio/" - | sed "s/printf/std::printf/" -
/* File: carne2.cpp
 * Info: The C++ version of carne.py
 */
#include 
 
typedef const char* cpstr;

int main(){
 /* string colors */
 cpstr red = "\x1b[1;31m";	/* red */
 cpstr grn = "\x1b[1;32m";	/* green */
 cpstr blu = "\x1b[1;34m";	/* blue */
 cpstr rst = "\x1b[0m";		/* required reset */
 /* printing a string */
 std::printf("%sHello %sColorful %sWorld%s\n",red,grn,blu,rst);
 return 0;
};

I can’t believe it took such a long explaination to state something that was so easy! We have what we are looking for. Let’s put it into it’s own file, compile it, and run it! (Note, the C compiler is gcc, and the C++ compiler is g++)

$ sed "1s/carne.c/carne2.cpp/" carne.c | sed "2s/C-version/C++ version/" - | sed "/include/s/stdio.h/cstdio/" - | sed "s/printf/std::printf/" - > carne2.cpp
$ g++ carne2.cpp -o carne2
$ ./carne2
Hello Colorful World

Gadzooks! We are almost done with this novel. We now get to go back to Python and do this. I know I was suppost to show you how to do that sed thing the other way, but now that I think if it, this way was way better. I’ll show you the other way when we use sed to make up lists.

So after reading all of this techie diatribe, I now come to the part that I was supposed to show you yesterday, but forgot about. Here’s a cleaned up version of the meat.py program.

$ cat meat.py
#!/usr/bin/python
# File: meat.py
# Info: Spam for carnivores!

# string colors
red = "\x1b[1;31m"      # red
grn = "\x1b[1;32m"      # green
blu = "\x1b[1;34m"      # blue
rst = "\x1b[0m"         # required reset

# printing a string
print "%sHello %sColorful %sWorld%s" % (red,grn,blu,rst)
$ python meat.py
Hello Colorful World

As you can see, print does all the processed work. I should have called this program processed_meat.py because it’s almost like you have to do nothing but put it on bread and eat it. Let’s get some raw meat and cook it this time around. The program steak.py will show us how to take a C function and use it in Python.

$ sed -e "2s/meat/steak/" -e "/Info/a import ctypes\nlibc = ctypes.CDLL('libc.so.6')" -e "/^print /{s/^print /libc.printf(/;s/\" % (/\\\\n\",/}" meat.py > steak.py
$ python steak.py
Hello Colorful World

That last sed command was a bit tricky. Four consecutive backslashes are needed so that the first and thrid backslashes escape in bash, and the second backslash escapes the forth. (Special thanks to the folks at ##sed on FreeNode for helping me out with getting that to work.)

So after all that, we now see why Python uses print instead of ctypes.CDLL('libc.so.6').printf() and if we so choosed, we could use them in C++ and in Python.