Tuesday, June 26, 2018

Homebrew Made PHP Harder on OS X

With Brew version 1.5, Homebrew announced they were deprecating the PHP tap. There blog post about it seems to understate what really happened. They have migrated the formulas for PHP installation to ONLY install a standard version of PHP with "the most common extensions". If other extensions are required, they have to be installed using Pecl, Pear or compiled manually.

This reminds me of the days when I had to fix dependencies and compile everything. After pulling my hair out for a few hours and reverting the entire local directory twice, I decided to relax and figure out what I needed to do to upgrade to PHP 7.1 (finally).

A month before the announcement of Brew 1.5 I had installed PHP 7.1 on my MacBook with great success including all the additional extensions that I needed for development. When I tried that same thing on my desktop, Brew updated itself, migrated php56 to php@5.6 and I fell into the rabbit hole.

My current projects require the following extensions:

  • mbstring
  • gd
  • memcached
  • xdebug
  • mailparse

The "standard" PHP came with MbString and GD. Memcached and Mailparse were the main problems not to mention I consider Xdebug an absolute requirement for any development environment.


Before installing memcached, pkg-config has to be installed with
brew install pkg-config

To actually install memcache, the package manager PECL is used.
pecl install memcached

But it asks for the libmemcached directory. After several attempts and some Googling, I found that providing the actual path to the library was not correct. Just supply the path to Brew's working directory.
As the comment from mkoppanen states, the supplied directory is just a prefix for the test to find memcached.h.

Pecl states that it adds the extension to PHP but this is not true. To accomplish this I added a file to the conf.d directory to load the extension (for both FPM and CLI)

File: /usr/local/etc/php/7.1/conf.d/20-memcached.ini


Relative to memcache, mailparse ended up being easier. Again use PECL.
pecl install mailparse

And again PECL states that it adds the extension but this is still not true. It requires manually linking it to PHP.

File: /usr/local/etc/php/7.1/conf.d/20-mailparse.ini


The only difference with Xdebug is that it's a Zend extension so the linking is different.
pecl install xdebug

With the extension getting loaded into PHP with a ini file in the conf.d directory.

File: /usr/local/etc/php/7.1/conf.d/10-xdebug.ini
[xdebug] zend_extension="/usr/local/Cellar/php@7.1/7.1.19/pecl/20160303/xdebug.so"

The location of the ini files linking these extensions to PHP is the same as what Brew does as seen with the OPcache extension.

Also since I'm running Nginx not Apache, some changes to PHP-FPM had to happen to make things work. To alleviate permission problems, the user and group are set to me. Also the listen mode has to be set to read/write by everyone so Nginx can connect correctly.

user = davidfairbanks
group = staff
listen.mode = 0666

While all of this is a pain to stop working and figure it out, it's still better than having Apple completely wipe my development environment when updates comes out. Not to mention easier than having to compile everything from source code which is what I used to do.

My local development environment

  • OS X Sierra
  • PHP 7.1
  • Nginx 1.12.2
  • MariaDB 10.3.7
  • Memcached
  • Beanstalkd
  • Node 8.11.2

Thursday, May 24, 2018

Interactive Command Line Directory Picker

Like most developers, I work on multiple projects, and I'm lazy so I don't like to constantly typing in directory paths to change my location in terminal. I used to use a custom bash script to make this task easier but it proved to be hard to maintain and not flexible.

Since I am primarily a PHP developer, I wanted to make a script in PHP to handle the interactive logic bits of determining what project I am trying to navigate to. As an added feature, I wanted to not have to memorize all the possible paths that I've setup in the chooser. To this end, I can enter as little as the function name or the beginning portion of the option or the entire path option:

  • chdir
  • chdir juno
  • chdir juno.api
  • chdir juno.api.logs
If the path entered is the final destination (resulting in a string) then you're redirected to that location. If it's a option (resulting in an array) you're given a choice of where to go.

A simple bash function is created in my ~/.bash_profile. The function takes the first argument it receives and passes it directly to the PHP script. The PHP script then puts the resulting path choice into a temporary file. Then the bash script reads the file. With some additional time spent, I probably could have avoided the temporary file but I didn't feel it was worth it. The PHP script validates the path, so as long as the result in the temporary file is not empty, the bash function changes the current working directory with the cd command.

chdir() {
    echo '' >> /tmp/chdir
    php /Users/david/dev/directory_chooser $1
    rm /tmp/chdir
    if [ "${route}" != '' ]
        cd $route

Because this function is placed into my bash profile script, there is no need to prepend the call with a period so it will result in changing the current working directory. The custom bash script I used previously had to be executed with a dot space prepending it to actually change my working directory.

The bulk of the work happens in a PHP script called directory_chooser. The possible path options are coded into the $routes array. The name displayed is the key. If the value is a string, then it's the destination for that option. If the value of is an array, then it's a deeper set of options. The special case is when an option has no key (index of zero), it's presented as the Root. So choosing juno.api will result in a choice between the Root and logs.

// Command line colors
$colorReset="\033[0m";           # Text Reset
$colorCyan="\033[0;36m";         # Cyan
$colorBRed="\033[1;31m";         # Bold Red
$colorBGreen="\033[1;32m";       # Bold Green
$colorBCyan="\033[1;36m";        # Bold Cyan

// Available path options
$routes = [
    'juno' => [
        'home'   => '/Users/david/dev/juno-home/Juno-Home',
        'api'    => [
            'logs' => '/Users/david/dev/juno-api/Juno-API/storage/logs'
        'app'    => '/Users/david/dev/juno-app/Juno-App',
        'mobile' => '/Users/david/dev/juno-mobile/Juno-Mobile'
    'example' => '/Users/david/dev/example.com'

$input = (isset($argv[1])) ? $argv[1] : null;
$route = null;

// If the input is a dot notation value, get the path option desired
if($input !== null && strpos($input, '.') !== false) {
    $parts = explode('.', $input);

    $route = $routes;
    foreach($parts as $part) {
        if(is_array($route) && isset($route[$part])) {
            $route = $route[$part];
// If the input is provided but is not a dot notation
elseif($input !== null && isset($routes[$input])) {
    $route = $routes[$input];
} else {
    $route = $routes;

// Drill down in the path options until we get to a string path
while(!is_string($route)) {
    $names = array_keys($route);

    if(count($names) == 1) {
        $route = $route[$names[0]];

    foreach($names as $index => $name) {
        if($name === 0) {
            echo "{$colorCyan}\t{$index} - Root{$colorReset}\n";
        } else {
            echo "{$colorCyan}\t{$index} - {$name}{$colorReset}\n";

    echo "{$colorBCyan}Which path (numeric index):{$colorReset}\n";
    $handle = fopen('php://stdin', 'r');
    $line = fgets($handle);
    if(isset($names[(int)$line]) && $route[$names[(int)$line]]) {
        $route = $route[$names[(int)$line]];

// Make sure path actually exists
// On success, write the path to a temporary file so the bash script can get it
if(file_exists($route)) {
    echo "{$colorBGreen}Changing location to {$route}{$colorReset}\n";
    file_put_contents('/tmp/chdir', $route);
} else {
    echo "{$colorBRed}ERROR: The destination does not exist - $route\n";

The while loop makes it so that you can have path options as deep or as shallow as we want.

Because I like a pretty command line tools, I'm colorizing the PHP output with the $colorCyan and similar values. There are libraries to make command line choices like this easy, but I didn't want the overhead of maintaining links to a library, that's why I'm using the basic PHP command line input method of stdin.

The names have been changed to protect the innocent.

Monday, August 5, 2013

How to Securely Communicate with a Non-Secure Server

Security is a very important consideration when building web applications. In order to build trust with your client, they need to know that their information is not going to be compromised. Obviously the best practice is to have the application installed on a server that has a good SSL certificate and the application is only accessible via HTTPS. This is all well and good if you're building an application that will run on a server that you control.

But what if you're building a distributed web application that will be installed by your customers on their servers? I've found that the end clients are, in general, not very savvy when it comes to server setups and they have to be walked through every step. Adding additional requirements or steps to the installation process can just make their head spin. Obviously there are clients out there that are very security conscience and tech savvy and either already have a secure server or know exactly how to set one up; they are much easier to deal with.

Getting down to the nitty gritty, the most important items to protect from snoopers is login credentials. But how do you do this when credentials are being transmitted over a non-secure connection to their server because their server doesn't support HTTPS?

One option is to build your own encryption method. This is rather much of a false sense of security. While the actual information being transmitted is not plain text, it is typically very easy to decipher. I've employed character remapping, usually adding salts, techniques to do this. The two big downfalls of this method are that you are transmitting the key when the page loads and it's a fixed remapping so, for instance, t always equals Y.

Another option is to use a client side encryption schema that can then be decrypted by your application. Javascript plugins, such as SJCL or jscrypto. The problem with any encryption that happens on the client side is that the server has to transmit everything that is needed to encrypt the data, and in turn everything that is needed to decrypt the data. There are ways of getting around the inherent security issues, just look at how Braintree does it. But this is not easy or quick.

The best option if you have a secure server at your disposal, say for client authentication to make sure that your client still should be allowed to use your product, is to send critical communications to that server instead. The principle works like this. Because any communication sent over HTTPS is secure by nature, all you have to do is make sure that all sensitive communications are being routed through your secure server. So in practice, you can have the browser send credentials to the secure server via JSONP (to the HTTPS address) or simple form data. The JSONP is a much more user friendly method in my opinion. The secure server then stores the credentials temporarily (maybe as long as five seconds). Once the secure server has the information, it sends a signal back saying I got the information, you can proceed. The browser then sends a signal to their non-secure server to get the credentials from your secure server. This is done via a cURL request in PHP. Again because you are sending the request to the HTTPS address of your server, the entire communication is encrypted. Your server sends back a json encoded response for instance. So now their non-secure server has the credentials to do what every you need it to do and they were never sent over a non-secure connection.

I wouldn't recommend sending all communications through you server because of the extra traffic, but for login's and changing credentials, the hit wouldn't be that bad.

Your server can either store the credentials in a database or memory via APC. Either way, as soon as their server requests the credentials, wipe them from your server. You don't want to store unencrypted credentials on your server, it's just asking for a disaster.

Monday, July 15, 2013

PHP - Log Maintenance

Running a PHP file from cron is easy enough, simply specify the PHP bin path, the path to the PHP script and where to send the output of the script. The issue comes with how to manage the size of the log file. You can set a job to truncate the file as opposed to append it but that means you loose any of the log lines that were in it. A better option is to use one of the cron jobs to rotate the logs into an archive of some sort.

This script first checks when it was last run, if it's under an hour ago, it skips to the remainder of the job.

We then check the file size, if it's over 1MB it starts the process of archiving it and creating a blank replacement. To be sure we can know when the archived log file was created, we put the epoch time that it was created in the file name.

Every time this portion of the job runs, it also checks to see if there are any log files that are older than seven days.  Because we put the epoch time that the archive was created, this is a simple matter of getting the epoch time back out of the file name and comparing it to our cutoff value. If the file is older than cutoff date, we simply delete it then log that we did so.

// only check log files for maintenance once an hour at most
if($process->lastRan < (time() - 3600)) {
    $logFile = LOG_PATH.'/cron.log';
    // if log file is over 1MB, archive it and create a new one
    if(filesize($logFile) > (1024 * 1024)) {
        // put the epoch time into the name so we can tell when it was archived
        $logFileBk = 'cron.'.time().'.log';
        // duplicate the file with the new name
        copy($logFile, LOG_PATH.'/'.$logFileBk);
        // create a blank 
        file_put_contents($logFile, date('M d, Y H:i:s')."\t[cron_controller]\tTruncated log file - $logFileBk\n");
    // look for any log file that is oder than 7 days
    $end = strtotime('00:00:00 - 7 days');
    $list = scandir(LOG_PATH);
    foreach($list as $i) {
        // get the date date from the file name if it matches the correct form
        if(preg_match('/^cron\.([0-9]{10})\.log$/',$i,$matches) === 1) {
            // if the found date is older than 7 days ago
            if((int)$matches[1] < $end) {
                // delete the file
                echo date('M d, Y H:i:s')."\t[cron_controller]\tDeleted log file - $i\n";

This script could be put in either the beginning of the cron job or at the end. But if you are using the PHP Cron Controller that I outlined in this post, I highly recommend doing this log maintenance as early in the script as possible. If you do it at the end of the script, you could run into a situation where a job called from the PHP Cron Controller is working and writing lines when this maintenance is being done.