Why you should be using strace

Strace is a debugging utility for *nix systems, used to monitor the system calls used by a program, as well as any signals received by the program. Strace is useful for seeing what an application is doing under the hood, which can be vital to find subtle bugs in your code obscured by built-in functionality. I frequently use it with PHP to diagnose some types of errors as well as quickly find what causes scripts to hang (if you just want to see what’s slow in your code, I recommend profiling your code instead).

What are system calls? They are calls to low level functions provided by the OS. This includes functions like mmap, select, recvfrom, and more. This means you’ll frequently get a lot of output from strace, so it can be difficult to find the problem, but once you understand some of the system calls it’s much easier.

How To Use Strace

For example, I have the following file:

<?php
require 'cool.php';

Running this file with strace php test.php, I see some of the following output:

getcwd("/home/brandon", 4096) = 14
lstat("/home/brandon/./cool.php", 0x7fff18039d20) = -1 ENOENT (No such file or directory)
lstat("/usr/share/php/cool.php", 0x7fff18039d20) = -1 ENOENT (No such file or directory)
lstat("/usr/share/pear/cool.php", 0x7fff18039d20) = -1 ENOENT (No such file or directory)
lstat("/home/brandon/cool.php", 0x7fff18039d20) = -1 ENOENT (No such file or directory)
getcwd("/home/brandon", 4096) = 14
lstat("/home/brandon/cool.php", 0x7fff1803be80) = -1 ENOENT (No such file or directory)
open("/home/brandon/cool.php", O_RDONLY) = -1 ENOENT (No such file or directory)

Here I can see all of the places PHP is looking for the file, and in which order. This helps me see that PHP is failing to find my file cool.php (this is a contrived example as PHP will throw a fatal error telling you this anyways). A better example of when this may be useful is with the gettext extension. I had an issue a while back where PHP wasn’t loading my language files. The code that actually loads the language file is buried in the extension, and invisible to me. However, using strace, I was able to see where PHP was looking for the language files, and correct the issue with my code.

Attaching Strace To A Running Process

Far more useful than starting a program with strace is attaching strace to a running process such as Apache or PHP-FPM. This can be very useful to see why a process is hanging or very slow.

strace -p PROCESS_ID

This will attach strace to the given process id. You’ll probably have to be root for that to work. This will continually show you strace output as long as the process is running.

A very useful alias I have is straceall:

straceall () {
  ps -ef | grep $1 | awk '{ print "-p " $2}' | xargs strace -v -s 100
}

This takes a process name as an argument, and attaches strace to every process with that name. This lets me attach strace to all php-fpm processes for example, by running:

straceall php-fpm

I have the alias in my .bashrc file.

Long running PHP script gotcha

So I was recently answering a question on StackOverflow, and learned something quite interesting about PHP.  This particular aspect of PHP mostly affects long running scripts, such as PHP based servers or daemons.

PHP does not re-use resource ids internally, so eventually your script could run into an error with the resource ID overflows and wraps around into the negatives, which pretty much breaks everything. There was a bug report opened about it in 2009, but the issue is still present. This is a huge issue, because resources are used all over, even in not-so-apparent places. For example, most file operations use resources, as do database connections and network sockets. Some of these are obvious, as functions return a resource object (such as socket_accept). Others are not obvious, like file_get_contents, which actually uses two resources under the hood, incrementing the internal resource ID counter by two

The maximum resource id depends on your architecture (32 or 64 bit), and possibly the options used to compile PHP. On 32 bit systems, it’s almost certainly going to be 2,147,483,647. On 64 bit systems, its probably going to be 9,223,372,036,854,775,807.

While this limit is far easier to hit on 32 bit systems, it’s still possible on 64 bit systems if your script is very long running. This limit is good to be aware of when designing your application, as it’s possible to mitigate by using techniques such as forking and multiple processes.

Demonstration:

<?php
echo gmp_init("0x41682179fbf5")."\n";// Resource id #4
echo gmp_init("0x41682179fbf5")."\n";// Resource id #5
echo gmp_init("0x41682179fbf5")."\n";// Resource id #6
file_get_contents('/etc/hosts');
echo gmp_init("0x41682179fbf5")."\n";// Resource id #9
echo gmp_init("0x41682179fbf5")."\n";// Resource id #10
echo gmp_init("0x41682179fbf5")."\n";// Resource id #11

The above code will demonstrate resource IDs incrementing if you run it locally (requires php5-gmp). You can also see file_get_contents incrementing the ID by two.

WordPress Must-Use Plugins

At this point in time, WordPress remains the easiest-to-use “CMS” available to us, so I end up developing quite a few sites with it. Somehow, there is this great feature I’ve never heard of until a few weeks back, so I thought I’d share with the class.

WordPress has this feature called “Must Use Plugins“, aka mu-plugins. They go in the directory /wp-content/mu-plugins/ which does not exist by default. WordPress will load every .php file in that top level directory, before regular plugins. In my efforts to write less crappy code on WordPress, this directory has proved invaluable. However, rarely do you want a bunch of .php files in that directory. Normally, you’d want a system more like the regular plugin directory where each plugin can be a file OR a directory.

I wrote up a little proxy loader script that accomplishes this. Just save as /wp-content/mu-plugins/proxy-loader.php or something, and it will load either /wp-content/mu-plugins/<plugin-name>/<plugin-name>.php OR /wp-content/mu-plugins/<plugin-name>/plugin.php if those files exist. The plugin is available via GitHub.

Writing nicer console scripts

Recently I working on a web application for a client over at my new company, Spark Creek. The particular component I was building was an import script, runnable via the command line. It was written in PHP, as a bundle for Symfony. At first, I built the script to output progress like “Matched 50 URLs on page: 1″. It would output a newline for every page. This produced a lot of output for a simple console script, and wasn’t very nice. What I wanted to do was match certain utilities like wget, which have a progress indicator that changes dynamically. I looked and couldn’t find how to do it, but with a little research I was able to recreate the effect.

The \x08 hex character code is the ASCII backspace character. If you write it to the console, it deletes a character. With PHP, I kept track of the line length, and every iteration of the loop, I would use str_repeat to send however many backspace characters I needed to erase the current line. Then I would write a new line for the current progress of the loop. This created a very smooth progress indicator.

Here’s a barebones version as a proof of concept:

<?php
$lastLineSize = 0;

for ($i = 0; $i <= 50; $i++) {
    echo str_repeat("\x08", $lastLineSize);
    echo "Iteration {$i} of 50";
    $lastLineSize = strlen("Iteration {$i} of 50");
    usleep(100000);
}

echo " - Done\n";

and the resulting console output after the script is complete looks like this:

[root@dev /home/brandon] > php test.php
Iteration 50 of 50 - Done
[root@dev /home/brandon] >

Run the script for yourself to see what it looks like. Basically, the script counts from iteration 0 to 50, all on one line. It makes for very fancy console scripts :)

web_debug_toolbar

Symfony: Using the current user object with a form

Recently I’ve been working with Symfony, a powerful and popular PHP framework. I’ve been running into my fair share of problems with the framework, a lot of which stems from lack of content/documentation for Symfony 2.x, so I’ve decided to start writing some blog posts about it.

For the application I’ve been developing (For one of my clients over at Spark Creek), I’ve had to create an edit profile page. Using Symfony’s form builder, I created my own form class but wanted to use the current user object instead of creating a dummy model. The problem I was running into was that whenever I tried changing my email address to one already in use, it would show me an error but it would switch by user account to the user with the email address I entered.

After a bit of research, I discovered that the current user object was being re-saved to the session, and since forms bind data to the entity/model even if it’s invalid, the new email was getting saved. Not to the database, just to the session. A bit more research lead me to a simple fix, if the form validation failed, call refresh() from the Doctrine entity manager. This function restores an entity to it’s original state (From the database). Continue reading…