Idle thoughts and technical knots.

Setup Laravel queue on shared hosting

We specialise in bespoke web and mobile development at Papertank, and we’re proud to use Laravel as our PHP framework of choice on web and API projects.

When it comes to hosting a Laravel application, installing on a shared host is perfectly possible, although we normally use a more powerful VPS server for more flexibility and control. We’ve previously blogged about setting up installing Laravel on shared hosting on papertank.co.uk. If you do go down the shared hosting route, you’ll want to make sure you are using a decent web hosting company. This will save a lot of headache and make deploying custom apps much easier. We recommend Cloud Above in Cornwall for great value and support.

Modern web apps use queues for background job handling of intensive tasks, allow your API or front-end app to return data, redirect the user, or perform other actions without waiting for the ‘job’ to finish. Laravel’s driver based queue system makes this easy to set up with Beanstalkd, Amazon SQS or Redis. Unfortunately shared hosting makes using these services difficult or impossible, since your host will often forbid installing software or logging in as root.

Laravel 5’s new database driver fills in for other tools, storing the jobs on a database table – https://laravel.com/docs/5.2/queues. The next step is running or listening for jobs using the command line ‘artisan’ tool, which can be a little tricky on shared hosting without process monitors. To get around this, you can use a cron job which can usually be setup in cPanel on shared hosting, however cron jobs can only be run once per minute which isn’t ideal for high traffic websites.

As a workaround, we recently customised a Laravel web app using the built in Task Scheduler. By setting up the server’s cron to run the schedule:run command every minute, our application spawns off a new queue:listen process and checks to see if it needs restarted every 5 minutes.

/**
* Define the application's command schedule.
*
* @param IlluminateConsoleSchedulingSchedule $schedule
* @return void
*/
protected function schedule(Schedule $schedule)
{
	$path = base_path();
	$schedule->call(function() use($path) {
		if (file_exists($path . '/queue.pid')) {
			$pid = file_get_contents($path . '/queue.pid');
			$result = exec("ps -p $pid --no-heading | awk '{print $1}'");
			$run = $result == '' ? true : false;
		} else {
			$run = true;
		}
		if($run) {
			$command = '/usr/bin/php -c ' . $path .'/php.ini ' . $path . '/artisan queue:listen --tries=3 > /dev/null & echo $!';
			$number = exec($command);
			file_put_contents($path . '/queue.pid', $number);
		}
	})->name('monitor_queue_listener')->everyFiveMinutes();
}

The full code for this setup is on Gist at https://gist.github.com/davidrushton/b7229df4c73372402fc1

If you’re looking for more help with Laravel, you should check out the following websites which we subscribe to and read regularly.

Of course, if you’re looking for a Glasgow based company with experience building advanced web apps with Laravel then please get in touch

Zero Downtime PHP Deployments

Modern PHP Deployments

We do a lot of bespoke web application development at Papertank, and we specialise in PHP for our server-side web and API projects.

PHP has come along way, and the whole platform has matured significantly in the last 5 years. In addition to core improvements and standards initiatives, the introduction of the Composer Dependency Manager (getcomposer.org) has been instrumental in allowing developers to share and ‘require’ other open-source projects within their applications.

We use Composer on our bespoke web projects, allowing us to install, update and manage all of the packages and libraries our websites and APIs use.

The Dreaded Downtime

Inevitably all software needs updated, and updating can be a pain. We use Git version control repositories and Composer, but sometimes things go wrong, and even when they don’t they can take a website offline for several minutes while the code and database is updated.

Our Zero Downtime Solution

A few months ago, we picked up on a server technique to deploy projects with zero downtime – effectively by linking the web root directory to the most current ‘copy’ of the application / code.

Using Laravel’s Envoy Task Runner, we then developed an automated process of initialising, running and cleaning up your deployments via the command line on your local command line. Envoy runs a series of commands on your remote host and restarts your web server, effectively resulting in no downtime when deploying complex updates including database migrations and composer updates.

Our code lives on Github at https://github.com/papertank/envoy-deploy and is specifically designed for use on Laravel 5 projects. However, the idea can be expanded for any PHP or non-PHP projects, and the Envoy command line runner can be used independent of Laravel.

How it Works

Your server will look something like this after you init and then deploy with the script

20150317110501/
20150317114500/
current -> ./20150317114500
storage/
.env

 

As you can see, the current directory is symlinked to the latest deployment folder

Inside one of your deployment folders looks like the following (excluded some laravel folders for space)

app/
artisan
boostrap/
composer.json
.env -> ../.env
storage -> ../storage
vendor/

 

The deployment folder .env file and storage directory are symlinked to the parent folders in the main (parent) path.

Websites we now use this approach on have 100% uptime.

Find out More

Our GitHub project – https://github.com/papertank/envoy-deploy

Not comfortable with the command line? – https://envoyer.io is a great service that offers the same feature through a web interface

Servers For Hackers did a great video showing a similar setup in more detail – https://serversforhackers.com/video/enhancing-envoy-deployment

Are you a client interested in a no-downtime web application? Get in touch

 

Why we’re building our own e-commerce software

When it comes to building e-commerce websites, it seems like there is a never ending list of options to choose from. All you have to do is Google “Ecommerce software” to get over 25 million results, from basic off-the-shelf templates to fully fledged systems targeted at enterprise customers.

We’ve been designing and developing e-commerce websites at Papertank since we first set up shop in 2010. In that time we’ve reviewed, used and customised a variety of e-commerce systems as well as coding bits and bobs of our own to suit the project.

Top E-commerce solutions

While there is a lot of good e-commerce software around, there is also a lot of bad and ugly ones – either because of sloppy coding, very out-of-date structure, or both.

Although each project is different, we often look at the following software for each e-commerce project before recommending the best solution to our clients.

Magento

Now owned and funded by eBay, Magento is without a doubt one the most popular e-commerce solutions. Commercially, Magento costs $15k per year just for the software. However, a free open-source version is also available with the same core functionality, albeit with less features and the expectation you know exactly what you’re doing or employ a development studio (like Papertank) to set up your website.

Magento is one of the most powerful and feature rich solutions – which is both a blessing and a curse. While customising the design and structure of the website is easy with experience, adding to the functionality can be very pricey – either though purchasing bolt-on third party extensions or coding custom features from scratch. As a “heavy” system, one of Magento’s biggest complaints is also the the slow speed and steep learning curve of doing day-to-day updates and configuration.

Shopify

Unlike Magento which you download and setup on your own, Shopify is a monthly subscription service which costs from $14 and upwards to use. This means that you have to pay a monthly subscription for using the software rather than setting up on your own server / hosting.

That said, Shopify is a wonderful service and is constantly adding new features including the ability to use their software as both an online shop and a face-to-face point of sale in your brick-and-mortar store.

WordPress

A quick caveat about WordPress – it is not e-commerce software and therefore third-party plugins need to be installed to get a typical product, cart, checkout facility working. While we use WordPress as our preferred content management system at Papertank, we don’t normally recommend its use for e-commerce unless there are specific requirements or the shop component is relatively basic.

For clients, the main attraction to using WordPress is using the one interface for both adding website content and managing orders, reducing the learning curve.

Why build our own?

With so many existing e-commerce systems out there, why decide to build our own from scratch? The answer is a very simple one – flexibility.

As a bespoke web studio, our clients and projects are not only varied but are often  both complex and challenging. Given the complex nature of e-commerce development, we want our projects to meet every requirement, whether it is as basic as a comment field on each order or as complex as a weekly lottery system with direct debit or credit card payments.

Coming soon…

Our as yet un-named e-commerce software is currently being planned, designed and implemented for one of our favourite clients – the Wild Futures animal charity.

After discussing and detailing what features their new shop should have, it became clear that using an off-the-shelf system was not going to work or, at best, would require a heavy and costly amount of customisation. Our bespoke solution will handle donations, adoptions, event tickets and merchandise sales, and we look forward to working with Wild Futures on this exciting new project.

If you’re interested in chatting about how our custom e-commerce system might work for your business, please don’t hesitate to get in touch with us.

 

 

Setup Laravel on shared hosting

Hosting Laravel

We’ve always specialised in PHP development at Papertank, but for the past 12 months we’ve been exclusively using the excellent Laravel framework for our bespoke web projects.

Laravel is relatively straightforward to get set up, particularly since it utilises  the new PHP dependency manager, Composer. For a quick start guide see the Laravel Docs.

Installing on a shared host is perfectly possible, although we normally use a more powerful VPS server for more flexibility and control. You’ll want to make sure you are using a decent web hosting company. This will save a lot of headache and make deploying custom apps much easier. We recommend South West Broadband in Cornwall for great value and support.

Installing on shared hosting

If you decide to use shared linux hosting, setting things up can be a little more footery, but following these steps should hopefully help fix main two issues:

Requirements

Laravel only has two main requirements which you’ll need to make sure your shared server meets:

  • PHP >= 5.4
  • MCrypt PHP Extension

Before you go any further however, you’ll also need to check (or ask) if your web server and hosting company also allows:

  • SSH shell access
  • PHP 5.4 command line interface

Public Path

If you’re installing and using Laravel correctly, you’ll be forced to put all the public content of your website (including js, css and img assets) into a /public directory. When using cPanel on a shared linux server, however, you will normally put your files in a public_html directory which is already accessible publicly.

Placing the Laravel files in your public_html directory has two consequences:

  • All of your sensitive files (including temporary files) might be openly accessible through the web browser
  • Your web addresses will be ugly, since they might look like http://example.com/public/contact

The easiest way (we think) to fix this is simply:

  1. Install your Laravel app in the directory above public_html inside a new folder of your choosing, e.g. code.
  2. Move the index.php file and .htaccess files from the laravel public folder to your normal public_html directory.
  3. Update the index.php file to reflect the path to your new code folder (or whatever you called it in step 1)

For example, from the normal index.php file (GitHub) we now have the following:

<?php
/**
* Laravel - A PHP Framework For Web Artisans
*
* @package Laravel
* @author Taylor Otwell <taylorotwell@gmail.com>
*/

/*
|--------------------------------------------------------------------------
| Register The Auto Loader
|--------------------------------------------------------------------------
|
| Composer provides a convenient, automatically generated class loader
| for our application. We just need to utilize it! We'll require it
| into the script here so that we do not have to worry about the
| loading of any our classes "manually". Feels great to relax.
|
*/

require __DIR__.'/../../code/bootstrap/autoload.php';

/*
|--------------------------------------------------------------------------
| Turn On The Lights
|--------------------------------------------------------------------------
|
| We need to illuminate PHP development, so let's turn on the lights.
| This bootstraps the framework and gets it ready for use, then it
| will load up this application so that we can run it and send
| the responses back to the browser and delight these users.
|
*/

$app = require_once __DIR__.'/../../code/bootstrap/start.php';

/*
|--------------------------------------------------------------------------
| Run The Application
|--------------------------------------------------------------------------
|
| Once we have the application, we can simply call the run method,
| which will execute the request and send the response back to
| the client's browser allowing them to enjoy the creative
| and wonderful application we have whipped up for them.
|
*/

$app->run();

SSH in to your server and run something similar to the below (replacing the italics with your values).

  cd /home/username/public_html
  ln -s /home/username/code/public/css css
  ln -s /home/username/code/public/img img

 

This creates an alias for the css and img folders from your laravel public directory inside the public_html directory.

Composer

Still with us? Load up your website in your browser and you’ll no doubt see a blank white page (if error reporting is off), or the following:

Warning: require(/home/username/code/bootstrap/../vendor/autoload.php): failed to open stream: No such file or directory in /home/username/code/bootstrap/autoload.php on line 17
Fatal error: require(): Failed opening required '/home/username/code/bootstrap/../vendor/autoload.php' (include_path='.:/opt/alt/php54/usr/share/pear:/opt/alt/php54/usr/share/php') in /home/username/code/bootstrap/autoload.php on line 17

This means we haven’t installed laravel (and its dependencies) via Composer.

SSH in to your server again and cd into the directory containing your laravel code (not public_html).

You’d normally install composer by running:

   curl -sS https://getcomposer.org/installer | php

 

But you’ll probably get the error

Some settings on your machine make Composer unable to work properly.
Make sure that you fix the issues listed below and run this script again:

Let’s fix this by creating a custom php.ini file in the same directory with the content from https://gist.github.com/davidrushton/8a65e6c194786bec881a

suhosin.executor.include.whitelist = phar;
disable_functions = none;
memory_limit = 512M;

 

Now if we run the command again specifying that we want to use this new php.ini file, composer should be installed correctly:

curl -sS https://getcomposer.org/installer | php -c ./php.ini

 

Now you want to install laravel fully by running the command

php -c ./php.ini composer.phar install

 

You can periodically run the following command to update the version of the laravel framework and other components.

php -c ./php.ini composer.phar update

Other Resources

If you’re looking for more help with Laravel, you should check out the following websites which we subscribe to and read regularly.

Of course, if you’re looking for a Glasgow based company with experience building advanced web apps with Laravel then please get in touch

 

How to fix: Codeigniter not adding products to the cart

Codeigniter is a great php framework, and the one of choice at Papertank. As with any framework code though, sometimes things just don’t work the way you want them to…

Hopefully this post will help any other programmers who come across the same issue – Codeigniter not adding particular products to the shopping cart. In my experience, this is caused by one of two reasons:

Problem A

The Codeigniter Shopping Cart library uses sessions to save your basket, and by adding lots of products or products with various options, this session can get exhausted pretty quickly.

Solution

The first thing to try, if you’ve not already, is to try switching to using your web app’s database for session storage. As well as allowing more information per session, this can increase security.

Before you go changing the config.php file, you’ll want to set up the right table in your database as so:


CREATE TABLE IF NOT EXISTS  `ci_sessions` (
	session_id varchar(40) DEFAULT '0' NOT NULL,
	ip_address varchar(45) DEFAULT '0' NOT NULL,
	user_agent varchar(120) DEFAULT NULL,
	last_activity int(10) unsigned DEFAULT 0 NOT NULL,
	user_data text DEFAULT NULL,
	PRIMARY KEY (session_id),
	KEY `last_activity_idx` (`last_activity`)
);

Now, you’ll want to update the config.php file, usually found in application/config/config.php so that the session library uses the database option. Also, just incase you decided to use a different name for the database table, you can chance this here too.

$config['sess_use_database'] = TRUE;

Problem B

If that didn’t fix your problem, check your error log for more information. For me, there was an issue with product names having invalid characters – in my case a ‘.’

ERROR - 2012-06-26 09:51:27 --> An invalid name was submitted as the product name: Urinal Blocks 1.1Kg The name can only contain alpha-numeric characters, dashes, underscores, colons, and spaces

Solution

To fix this problem, you’ll need to overwrite the default Shopping Cart class – CI_Cart – so that it no longer rejects particular names. In your application/libraries folder, add a file called MY_Cart.php with the following code, or download from https://gist.github.com/16f99e4a38386fac461e:


<?php if (!defined('BASEPATH')) exit('No direct access allowed.');

class MY_Cart extends CI_Cart {
 
  public $CI;

  function __construct()
  {
    parent::__construct();

    $this->CI =& get_instance();

  } 
  
	function format_number($n = '')
	{
		if ($n == '')
		{
			return '';
		}
	
		// Remove anything that isn't a number or decimal point.
		$n = trim(preg_replace('/([^0-9.])/i', '', $n));
	
		return number_format($n, 2, '.', '');
	}
	
	/**
	 * Insert
	 *
	 * @access	private
	 * @param	array
	 * @return	bool
	 */
	function _insert($items = array())
	{
		// Was any cart data passed? No? Bah...
		if ( ! is_array($items) OR count($items) == 0)
		{
			log_message('error', 'The insert method must be passed an array containing data.');
			return FALSE;
		}

		// --------------------------------------------------------------------

		// Does the $items array contain an id, quantity, price, and name?  These are required
		if ( ! isset($items['id']) OR ! isset($items['qty']) OR ! isset($items['price']) OR ! isset($items['name']))
		{
			log_message('error', 'The cart array must contain a product ID, quantity, price, and name.');
			return FALSE;
		}

		// --------------------------------------------------------------------

		// Prep the quantity. It can only be a number.  Duh...
		$items['qty'] = trim(preg_replace('/([^0-9])/i', '', $items['qty']));
		// Trim any leading zeros
		$items['qty'] = trim(preg_replace('/(^[0]+)/i', '', $items['qty']));

		// If the quantity is zero or blank there's nothing for us to do
		if ( ! is_numeric($items['qty']) OR $items['qty'] == 0)
		{
			return FALSE;
		}

		// --------------------------------------------------------------------

		// Validate the product ID. It can only be alpha-numeric, dashes, underscores or periods
		// Not totally sure we should impose this rule, but it seems prudent to standardize IDs.
		// Note: These can be user-specified by setting the $this->product_id_rules variable.
		if ( ! preg_match("/^[".$this->product_id_rules."]+$/i", $items['id']))
		{
			log_message('error', 'Invalid product ID.  The product ID can only contain alpha-numeric characters, dashes, and underscores');
			return FALSE;
		}

		// --------------------------------------------------------------------

		// Validate the product name. It can only be alpha-numeric, dashes, underscores, colons or periods.
		// Note: These can be user-specified by setting the $this->product_name_rules variable.
		/*
if ( ! preg_match("/^[".$this->product_name_rules."]+$/i", $items['name']))
		{
			log_message('error', 'An invalid name was submitted as the product name: '.$items['name'].' The name can only contain alpha-numeric characters, dashes, underscores, colons, and spaces');
			return FALSE;
		}
*/

		// --------------------------------------------------------------------

		// Prep the price.  Remove anything that isn't a number or decimal point.
		$items['price'] = trim(preg_replace('/([^0-9.])/i', '', $items['price']));
		// Trim any leading zeros
		$items['price'] = trim(preg_replace('/(^[0]+)/i', '', $items['price']));

		// Is the price a valid number?
		if ( ! is_numeric($items['price']))
		{
			log_message('error', 'An invalid price was submitted for product ID: '.$items['id']);
			return FALSE;
		}

		// --------------------------------------------------------------------

		// We now need to create a unique identifier for the item being inserted into the cart.
		// Every time something is added to the cart it is stored in the master cart array.
		// Each row in the cart array, however, must have a unique index that identifies not only
		// a particular product, but makes it possible to store identical products with different options.
		// For example, what if someone buys two identical t-shirts (same product ID), but in
		// different sizes?  The product ID (and other attributes, like the name) will be identical for
		// both sizes because it's the same shirt. The only difference will be the size.
		// Internally, we need to treat identical submissions, but with different options, as a unique product.
		// Our solution is to convert the options array to a string and MD5 it along with the product ID.
		// This becomes the unique "row ID"
		if (isset($items['options']) AND count($items['options']) > 0)
		{
			$rowid = md5($items['id'].implode('', $items['options']));
		}
		else
		{
			// No options were submitted so we simply MD5 the product ID.
			// Technically, we don't need to MD5 the ID in this case, but it makes
			// sense to standardize the format of array indexes for both conditions
			$rowid = md5($items['id']);
		}

		// --------------------------------------------------------------------

		// Now that we have our unique "row ID", we'll add our cart items to the master array

		// let's unset this first, just to make sure our index contains only the data from this submission
		unset($this->_cart_contents[$rowid]);

		// Create a new index with our new row ID
		$this->_cart_contents[$rowid]['rowid'] = $rowid;

		// And add the new items to the cart array
		foreach ($items as $key => $val)
		{
			$this->_cart_contents[$rowid][$key] = $val;
		}

		// Woot!
		return TRUE;
	}
	 
}

If you look closely, you’ll notice what we’ve done here is actually comment out the specific section that validates the product name. Be careful with this, of course, but when you re-try adding your product, hopefully your issue should be solved.