Shuffling shell input

April 17th, 2009 2 comments

While the Fisher-Yates shuffle sounds like 1920’s dance move, it is the standard method for shuffling larger arrays of data.

The below operates like a simple Unix filter taking an array of data (separated by newlines) and returning a quickly sorted output.

#!/usr/bin/perl
# Usage: cat file | shuffle
# From: http://hashbang.net/2009/04/shuffling-shell-inputshuffling-shell-input/
# Return a shuffled version of the input stream
# Author: Matt Carter 

sub fisher_yates_shuffle {
	my $array = shift;
	my $i;
	for ($i = @$array; --$i; ) {
		my $j = int rand ($i+1);
		next if $i == $j;
		@$array[$i,$j] = @$array[$j,$i];
	}
}

while(<>) {
	chomp;
	push @input, $_;
}

&fisher_yates_shuffle(\@input);
print join("\n",@input);

For example:

ls | shuffle

Returns a randomized list of files in the current directory.

Categories: HowTo's Tags: ,

Easy Apache VirtualHost config

March 15th, 2009 1 comment

Like most of the word I use Apache to serve up both my live and test websites.

The downside to having a box for development though is the constant config of apache to host these sites, most of which require absolute links (e.g. ‘/css/style.css’) to function.

The Apache snippet below sets up dynamic sub-domains which can be changed without reloading Apache every time you add, remove or update anything something. Simply drop your sites inside /var/www whenever you want a new subdomain. Alternatively you can symlink if they are hosted elsewhere on the box.

Its pretty easy to install, simply paste the below in your /etc/apache2/apache.conf file…

# Easy virtual host config
# See http://hashbang.net/2009/03/easy-apache-virtualhosteasy-apache-virtualhost/

DocumentRoot /var/www/
ServerAdmin admin@localhost
Options +FollowSymLinks -Indexes
LogLevel debug
ServerAlias localhost
ServerName localhost
RewriteEngine  on
RewriteLogLevel 3
RewriteCond    %{HTTP_HOST}  ^localhost
RewriteRule    ^(.*)$        /www/$1 [L]
RewriteCond    %{HTTP_HOST}  ^www.*
RewriteRule    ^(.*)$        /www/$1 [L]
RewriteCond    %{HTTP_HOST}  ^(.*)\.localhost
RewriteRule    ^(.*)$        /%1/$1 [L]

Now add your servers to the /etc/hosts file:

127.0.0.1      server.localhost

Now any directory you place in the /var/www directory automatically becomes a subdomain. You can now visit http:///server.localhost as a fully fledged webhost.

If you want to use another name than the rather boring ‘localhost’ simply replace the text in both text pastes above.

If only there were a way of removing that /etc/hosts hack it would mean zero config from then on. If anyone has any ideas do let me know. The addition to the hosts file seems to need applying if you are working from the same box the Apache server resides on. If you arn’t and the box is a local dedicated server its not needed at all and domains can be added and removed as necessary.

Categories: HowTo's Tags: , ,

Trash with Bash

March 4th, 2009 6 comments

One of the regrettably unavoidable aspects of the Unix shell like environments is the impedance between what a user says and what a user means. While this is present in all computing environments the sheer power of Unix based command lines make a potential mistake catastrophic.

Who amongst us has not at some point done something like:

> rm * .tmp

or

> rm *>tmp

The former makes the mistake of a space between the dot and the ‘tmp’ part and the latter the accidental holding down of the shift key while pressing the intended full stop.

While searching for an alternative for the I-have-done-this-too-many-times-and-now-its-embarrassing approach of nuking everything with the ‘rm’ command its time i did something about it.

Inserting the following in your ~/.bashrc file will remap the ‘rm’ command to a slightly safer move-to-trash like behavior. Its not comprehensive but it has saved my life on more than one occasion.

# Trash support by Matt Carter /dev/null

Now the command ‘rm’ moves files into ~/.trashcan. You can also use the command ‘rm!’ when you really mean delete immediately and the utility command ’emptytrash’ to clean everything out.

Categories: HowTo's Tags: ,

Error reporting with CodeIgniter

February 23rd, 2009 5 comments

While CodeIgniter does come with a reasonable error logging tool its simple nature of just dumping a single line report to a file on a distant server does not seem too pro-active for my tastes.

The following helper replaces the default CI error reporting with a new error reporting interface that also emails any errors that occur to a nominated developer.


* Info: http://hashbang.net/2009/02/error-reporting-with-cierror-reporting-with-ci/
*/

// Where to send the error report. Leave blank for nowhere
define('ERR_MAIL_TO', 'someone@somewhere.com');

 // Literal name of that person
define('ERR_MAIL_TO_NAME', 'Joe Random');
define('ERR_MAIL_FROM', 'root@freesweetsite.com');
define('ERR_MAIL_FROM_NAME', 'Error Daemon');
define('ERR_MAIL_METHOD', 'SMTP'); // Supported: SMTP
define('ERR_MAIL_HOST', 'localhost');
define('ERR_MAIL_SENDER', 'root@somewhere.com');

 // The subject of the mail. Supported tags: [TYPE], [FILE], [LINE], [STRING], [BASENAME]
define('ERR_MAIL_SUBJECT', 'Error detected: [TYPE] @ [BASENAME]:[LINE]');
 // Other things of interest to include in the email. CSV of supported values: POST, GET, SERVER, GLOBALS, SESSION
define('ERR_MAIL_FOOTERS', 'POST,GET,SESSION');

// Relative location of PHP mailer to this file
// (Not relative to the working directory because that doesn't specify correctly with fatal errors).
// By default this assumes that the 'phpMailer' folder is located in the same directory as this script.
define('ERR_PATH_PHPMAILER', 'phpMailer/class.phpmailer.php');

function err($errno, $errstr, $errfile, $errline, $errcontext) {
	$errtable = array(
		1 => 'Fatal',
		2 => 'Warning',
		4 => 'Parse Error',
		8 => 'Notice',
		16 => 'Core Error',
		32 => 'Core Warning',
		64 => 'Compile Error',
		128 => 'Compile Warning',
		256 => 'User Error',
		512 => 'User Warning',
		1024 => 'User Notice',
		2048 => 'Strict Notice',
		4096 => 'Recoverable Error',
		8192 => 'Deprecated',
		16384 => 'User Deprecated',
	);
	$message = "";
	$message .= "
"; $message .= ""; $message .= ""; $message .= ""; $message .= ""; $message .= ""; $message .= ""; $message .= "
Type:{$errtable[$errno]}
Error:$errstr
File:$errfile
Line:$errline
Context:$errcontext
"; $traces = debug_backtrace(); array_shift($traces); if ( (count($traces) > 1) && ($traces[0]['function'] != 'err_shutdown') ) { // Ignore fatal shutdown traces $message .= ""; foreach ($traces as $offset => $trace) { // Calculate line number if ($offset == 0) { // First in trace $message .= ""; } $message .= "
LineFunction
$errline"; } else // Nth in trace $message .= "
" . (isset($trace['line']) ? $trace['line'] : ' ') . ""; // Calculate arg stack $trace['argstack'] = ''; if (isset($trace['args']) && $trace['args']) { foreach ($trace['args'] as $arg) $trace['argstack'] .= _err_human($arg) . ' , '; if ($trace['argstack']) $trace['argstack'] = substr($trace['argstack'], 0, -3); } // Output context if (isset($trace['object'])) { // Object error $message .= "{$trace['class']}->{$trace['function']}({$trace['argstack']})"; } else // Function error $message .= "{$trace['function']}({$trace['argstack']})"; $message .= "
"; } $message .= "
"; if (ERR_MAIL_TO) { if (!file_exists($p = dirname(__FILE__) . '/' . ERR_PATH_PHPMAILER)) { echo "Cannot find PHP mailer at the relative path '" . ERR_PATH_PHPMAILER ."'. Make sure it is located there to send mails"; } else { require_once($p); $mailobj = new PhpMailer(); if (ERR_MAIL_METHOD == 'SMTP') $mailobj->IsSMTP(); $mailobj->IsHTML(TRUE); $mailobj->CharSet = 'utf-8'; $mailobj->Host = ERR_MAIL_HOST; $mailobj->Sender = ERR_MAIL_SENDER; $mailobj->From = ERR_MAIL_FROM; $mailobj->FromName = ERR_MAIL_FROM_NAME; $mailobj->AddAddress(ERR_MAIL_TO, ERR_MAIL_TO_NAME); $mailobj->Subject = strtr(ERR_MAIL_SUBJECT, array( '[TYPE]' => $errtable[$errno], '[FILE]' => $errfile, '[BASENAME]' => basename($errfile), '[LINE]' => $errline, '[STRING]' => $errstr, )); $extras = preg_split('/\s*,\s*/', ERR_MAIL_FOOTERS); if (in_array('GET', $extras)) $message .= _err_dump_array($_GET, 'Get'); if (in_array('POST', $extras)) $message .= _err_dump_array($_POST, 'Post'); if (in_array('SESSION', $extras)) $message .= _err_dump_array(isset($_SESSION) ? $_SESSION : array(), 'Session'); if (in_array('SERVER', $extras)) $message .= _err_dump_array($_SERVER, 'Server'); if (in_array('GLOBALS', $extras)) $message .= _err_dump_array($_GLOBALS, 'Globals'); $mailobj->Body = $message; $status = $mailobj->Send(); } } echo $message; return TRUE; } // Catch fatal errors function err_shutdown() { $err = error_get_last(); if (in_array($err['type'], array(E_ERROR, E_CORE_ERROR, E_COMPILE_ERROR, E_USER_ERROR))) err($err['type'], $err['message'], $err['file'], $err['line'], 'Fatal error'); } function _err_human($what) { if (is_object($what)) { return get_class($what); } elseif (is_array($what)) { return "Array[" . count($what) . "]"; } else return $what; } function _err_dump_array($array, $title) { if ($array) { $out = ''; $out .= ""; foreach (array_keys($array) as $key) $out .= ""; $out .= '
Dump of $title array
KeyValue
$key" . _err_human($array[$key]) . "
'; } else { $out = "
$title is empty
"; } return $out; } set_error_handler('err', E_ALL); register_shutdown_function('err_shutdown'); ?>

Installation of the component is similar to any CI helper: drop the file into the application/system/helpers directory and create the associated entry in the application/system/config/autoload.php file like so:

$autoload['helper'] = array('err');

Then open the above and change the constants at the top of the file to specify where the error messages are sent.

In order to send mail the above component needs the truly excellent PHPMailer package which can be stashed in the same helpers directory. If you do decide to move it somewhere else change the corresponding constant that specifies the location of the library. A small note of warning though – the path to the PHP mailer libraries must be relative to the current file NOT the current working directory. This is due to how PHP handles fatal errors which for some reason sets the working directory to ‘/’.

Should there be any interest I may extend the above with other useful features such as RSS, SMS (text messaging) or other non-SMTP methods of emailing.

Categories: PHP Tags: , ,

Talking in tongues – UTF8 with CodeIgniter

February 16th, 2009 14 comments

PHP 4 & 5 unfortunately have major problems working with UTF8. Hopefully this will be solved with PHP6 and its fancy pants new rendering interface.

Until then we have to make do and mend.

First setup your database to work with UTF8. I won’t waste time telling you how to do this in MySQL as many people far brighter than I have written more on the subject.

CodeIgniter comes with UTF8 enabled out of the box so there is little to do configuration wise. Unfortunately pasting a test string such as: 検索 (which I stole from this Wikipedia page) into a CI input box will quickly lead to calamity as CI tries to save the string as ASCII.

Simple solution then, save the following helper in your system/application/helpers folder:


* Info: http://hashbang.net/2009/02/utf8-with-codeigniter
*/
function ob_utf8($string) {
	return utf8_decode($string);
}
ob_start('ob_utf8');

foreach ($_POST as $key => $val) // Re-write all incoming text into UTF8 streams
	$_POST[$key] = utf8_encode($val);
?>

Then simply add it to your system/application/autoload.php:

$autoload['helper'] = array('utf8');

The helper will automatically convert incoming POST data into MySQL compatible UTF8 and convert outgoing text into HTML UTF8 streams.

Categories: PHP Tags: , , ,