Some helpers for CFWheels DBMigrate Plugin

Here is a handy controller that I use with my CFWheels application deployments & for debugging my database migrations with Troy Murray’s invaluable CFWheels DBMigrate Plugin.

I use a git branching strategy that utilises feature branches, which can sometimes lead to database migration files being out of sequence based on their creation date.. So I needed a way to test that all current migration are in the correct order… Enter the magical DBMigrate.cfc controller.

To get this up and running, You’ll need to install the CFWheels DBMigrate Plugin, the /models/DBMigrateVersion.cfc model file and the /controllers/DBMigrate.cfc controller from https://github.com/chapmandu/cfwheels-dbmigrate-plugin-tools.

Once these files are in place, you can call the following URLs and see the JSON they return:

http://yoursite/index.cfm?controller=dbmigrate&action=ping

{
db: "okay"
}

http://yoursite/index.cfm?controller=dbmigrate&action=migrations

{
totalMigrationCount: 4,
isOrdered: true,
latest: "20160205201245",
versions: [
{
version: 20140527111730,
cfc: "20140527111730_create_tables",
migrated: true,
details: "create tables"
},
{
version: 20160205201146,
cfc: "20160205201146_insert_lookup_rows",
migrated: false,
details: "insert lookup rows"
},
{
version: 20160205201233,
cfc: "20160205201233_add_plain_text_password_field_to_users_table",
migrated: false,
details: "add plain text password field to users table"
},
{
version: 20160205201245,
cfc: "20160205201245_create_credit_card_number_columns",
migrated: false,
details: "create credit card number columns"
}
],
current: 20140527111730,
migratedCount: 1,
notMigratedCount: 3,
isMigrated: false
}

http://yoursite/index.cfm?controller=dbmigrate&action=current

{
current: 20140527111730
}

http://yoursite/index.cfm?controller=dbmigrate&action=latest

{
latest: 20160205201245
}

http://yoursite/index.cfm?controller=dbmigrate&action=ismigrated

{
ismigrated: false
}

The main one I use for my automated tests is the big sexy action=migrations packet. If any of the migrations are not in the correct sequence, the isordered node in the JSON will have a value of false Here is the RocketUnit test that I run to check if all migration are in the right order (and can be run automatically).

public void function test_db_checks_awaiting_migration_files_should_be_in_order()
{
    loc.args.action = "migrations";
    loc.response = getResponse(argumentCollection=loc.args);
    loc.substring = '"isordered":true'
    assert('loc.response contains loc.substring');
}

Disclaimer: Please be careful leaving this controller lying around in your production environment.. It doesn’t perform any database changes, but can give some basic information about your database migrations to anyone who may stumble upon your URL..

If you use the DBMigrate plugin, you may like this blog post… If you don’t use the plugin.. You really should.

Create globally unique URLs within CFWheels applications

One great feature of the CFWheels framework is being able to override inbuilt functions, yet still be able to call the core function from within. This allows you to slightly tweak core functions without having to copy, paste & butcher core code, which would 1) go against the DRY principle, and 2) create possible legacy issues when the core functions are updated.

The example below tweaks the urlFor() function by adding a random token to all urls generated within a CFWheels application. This can be handy to avoid page caching as every url is unique.

Because the urlFor() function is used within linkTo(), redirectTo() and startFormTag(), this single tweak trickles down to all CFWheels functions that generate urls.

// controllers/Controller.cfc

/**
 * Hint: Appends a token to all urlFor() calls
 */

public any function urlFor() {

	// create your random string
	var randomToken = LCase(Replace(CreateUUID(), "-", "", "all"));

	// defaults
	param name="arguments.params" default="";
	param name="arguments.token" default="true";

	// add the token to the URL unless told otherwise.
	if (arguments.token) {
		arguments.params = ListAppend(arguments.params, randomToken, "&");
	}

	// pass any args to the core urlFor helper
	return core.urlFor(argumentCollection=arguments);
}

Your urls that once looked like this /mypage, now magically look like /mypage?d9e4bd5a3b78427dab33fe02e0c14b5a=

Easily build where clauses for the CFWheels ORM with this whereify helper

Rather than stuff around building up a string when one’s WHERE clause is conditional.. I make heavy use of this simple little helper which takes an array, and returns a finished string ready to pass to the CFWheels ORM.

// helpers.cfm

/**
 * Hint
 * I surround each array element in brackets and return delimited by an operator
 */
public string function whereify(required array array, string operator="AND") {
	var loc = {};
	loc.array = [];
	for (loc.i=1; loc.i <= ArrayLen(arguments.array); loc.i++) {
		loc.array[loc.i] = "(#arguments.array[loc.i]#)";
	}
	return ArrayToList(loc.array, " #arguments.operator# ");
}

And here’s how I use it..

// YourController.cfc

var loc = {};
// build a sexy where clause array..
loc.where = [];
ArrayAppend(loc.where, "foo = 'bar'");
if (StructKeyExists(params, "slayer")) {
	ArrayAppend(loc.where, "somecolumn IN (1,2,3)");
}

// use the whereify helper in your ORM call
// Produces: (foo = 'bar') AND (somecolumn IN (1,2,3))
users = model("User").findAll(whereify(loc.where));

// You can also specify an operator
// Produces: (foo = 'bar') OR (somecolumn IN (1,2,3))
users = model("User").findAll(whereify(loc.where, "OR"));

Deploying a Lucee application to an Amazon Elastic Beanstalk Environment

If there was a piece of interwebs kit I would like to buy flowers, it would without doubt be Amazon’s Elastic Beanstalk with autoscaling.. Couple it with the Lucee scripting language and I’d happily take it home to meet my parents..

After much experimentation and many iterations (failed attempts and hair-pulling), I’ve managed to deploy Lucee applications (with my fave framework.. CFWheels) to the AWS Elastic Beanstalk platform and have it running reliably in production for a number of months.

My setup is a bit more complex than described below, but hopefully these bare bones will get you started on the path to automated utopia.. Those bare bones will result in a finished .war file that you can upload to EB using their deploy by file upload feature. You could also use the AWS command line tools, AWS toolkit for Eclipse, a Jenkins plugin or one of the many SaaS deployment platforms around.

I personally use Jenkins (on an AWS EC2 instance) to pull, build, test and deploy. I prefer this type of setup as there is much less margin for human error, and the resulting 60Mb+ war file is transferred within your Virtual Private Cloud rather than travelling across the interwebs if you were to manually upload it for every deployment.. oh, and it’s open source.

At a high level, I will outline how to download the Lucee server war file, combine your application’s code, and use some AWS sorcery to configure your Lucee application. There are some pre-requisites:

  • The settings that you would normally configure in Lucee admin, should be in Application.cfc (See the Export feature in Lucee admin)
  • The commands to create the finished war file are written for Linux (If you’re Windows inclined , you could either ‘translate’ them to Windows commands, try Cygwin or a Virtual Machine)
  • You’ll need a src/ directory in your application containing the files outlined below
  • An AWS account with an empty Tomcat 8 Elastic Beanstalk environment

In a nutshell, what happens is the Lucee jars, your code and some config files are all copied to the “build” directory, then zipped into a sexy war file ready for upload/deploy to the Elastic Beanstalk environment.

src directory
src/eb-tomcat/web.xml
src/eb-tomcat/urlrewritefilter-4.0.3.jar
src/eb-tomcat/urlrewritefilter.xml
src/eb-tomcat/.ebextensions
src/eb-tomcat/.ebextensions/lucee-server.xml
src/eb-tomcat/.ebextensions/01-copy-lucee-server-xml.config

src-directory-structure

These are configuration files.. for a production application, you’ll need to provide your own lucee-server.xml and most likely your own rewrite rules in urlrewrite.xml. If you don’t need url rewriting, then remove all reference to them. The .ebextensions execute EB container commands.. the one provided simply copies your lucee-server.xml file into place. To create your finished war file, cd to your application’s root directory then run these commands. I’d suggest creating a shell script.. and if you feel adventurous, allow an ‘environment’ option that will create different environment builds for your application’s deployment pipeline

# some vars..
DOWNLOADS_PATH="/tmp/downloads/"
LUCEE_VERSION="4.5.1.000"
LUCEE_WAR="lucee-$LUCEE_VERSION.war"
LUCEE_WAR_PATH="$DOWNLOADS_PATH$LUCEE_WAR"
FINISHED_WAR_PATH="/var/www/myapp/war/"

# ensure the required packages are installed
sudo apt-get install unzip wget
# sudo yum install unzip wget # (or using yum)

# create required directories
mkdir build
mkdir -p $DOWNLOADS_PATH
mkdir -p $FINISHED_WAR_PATH
# only download the lucee war file once
if ! [ -e "$LUCEE_WAR_PATH" ]
then
  wget -O $LUCEE_WAR_PATH http://bitbucket.org/lucee/lucee/downloads/$LUCEE_WAR
fi

# unzip the lucee war file into the build utility directory
unzip $LUCEE_WAR_PATH -d build
rm -rf build/assets

# explicitly copy required code (this is cfwheels specific.. but just plonk in your own directory names)
cp -r config controllers customtags events images javascripts models plugins  stylesheets tests views wheels build
cp -r Application.cfc index.cfm rewrite.cfm root.cfm build
# remove any peksy files that you don't want deployed to production
rm -r build/favicon.ico build/License.txt
# copy config files
cp -r src/eb-tomcat/.ebextensions build
cp -f src/eb-tomcat/web.xml build/WEB-INF/web.xml
# copy tuckey url rewrite jar if required
cp src/eb-tomcat/urlrewritefilter-4.0.3.jar build/WEB-INF/lib/urlrewritefilter-4.0.3.jar
cp src/eb-tomcat/urlrewrite.xml build/WEB-INF/urlrewrite.xml

# this is how I force my app into 'production' mode.. again very cfwheels-centric
rm build/config/environment.cfm
echo 'set(environment="production");' > build/config/environment.cfm
# create le war file
cd build && zip -r -q $FINISHED_WAR_PATH"app-production-master.war" . && cd ..
# cleanup
rm -r build

This should give you a production ready war file located at: build/app-production-master.war. Now you can manually upload your app-production-master.war and be on your way..

Some of my recommendations:

  • Build in the cloud using Jenkins (or your choice of CI server)
  • Host the Lucee war file in your own S3 bucket.. you’re less at the mercy of 3rd parties connectivity issues.
  • You can easily incorporate additional jar files as per the tuckey rewrite tool.. just copy them into build/WEB-INF/lib/

Any suggestions, improvements & feedback welcome.

Enforcing the use of routes in CFWheels

If you really like routing (you’d find that really funny if you’re Australian) and want to enforce their use in all linkTo(), urlFor() and startFormTag() function calls.. Do as I do..

Put the function below into your /controllers/Controller.cfc

<!--- /controllers/Controller.cfc --->
<cffunction name="urlFor" access="public" output="false" hint="ensure routes are used for all urls">
    <cfif ! StructKeyExists(arguments, "noroute") && ! ( (StructKeyExists(url, "controller") && url.controller == "wheels") || StructKeyExists(arguments, "url") )>
      <cfif ! (StructKeyExists(arguments, "route") && Len(arguments.route) gt 0)>
        <cfthrow message="Please use a route rather than an action.. Thanks!">
      </cfif>
    </cfif>
    <cfreturn core.urlFor(argumentCollection=arguments)>
  </cffunction>
</code>

This overrides, the core urlFor() method, checking for the existence of a route argument.. UNLESS, the controller is “wheels” which probably means you’re using the test framework or managing a plugin. There is also a way to bypass the route nazi.. by using the noroute=true arguments.

Happy routing.. Teehee!

CFWheels Calculated Properties on Steroids using an afterFind Callback

One limitation with using calculated properties in your CFWheels models is that one is limited to what can be achieved using SQL. CFML has almost infinite possibilities for formatting strings which simply cannot be done easily (or at all) in SQL.

A technique that I use to get around this limitation is to create new properties using an afterFind callback for both objects AND queries in a single callback.

In the code below I create two new properties, createdAtShort and createdAtLong which are just different date masks applied to the trusty createdat property. It’s a fairly simple example, but highlights the huge potential of keeping this type of logic in your model and out of your views. Hopefully this will get your brain moist.

<!--- /models/User.cfc --->

<cfcomponent extends="Model" output="false">

  <cffunction name="init">
    <cfset afterFind("$$setStuffAfterFind")>
  </cffunction>

  <cffunction name="$$setStuffAfterFind" access="private">
    <cfif StructKeyExists(arguments, "createdAt")>
      <cfset arguments.createdAtShort = DateFormat(arguments.createdAt, "d/m/yy")>
      <cfset arguments.createdAtLong = DateFormat(arguments.createdAt, "dddd, dd mmmm yyyy") & " " & TimeFormat(arguments.createdAt, "hh:mmtt")>
      <cfreturn arguments>
    </cfif>
  </cffunction>

</cfcomponent>

Now you can call your model and the result should contain your new properties..

<!--- /controllers/Users.cfc --->

<cfset user = model("User").findOne()>
<cfset users = model("User").findAll(maxrows=5)>

<cfdump var="#user.properties()#">
<cfdump var="#users#">

Notes:

  1. Avoid performing further database operations inside the callback when using findAll.. y’know the whole querying within a recordset loop thing..
  2. afterFind callbacks are not called on included models. So.. model("Company").findAll(include="Users") won’t contain your new properties.

UPDATE: Chris Peters has since shown me the error of my ways.. it turns out that the arguments scope can be used for both queries and objects. This has simplified the entire concept considerably (and made ‘my’ code look eerily similar to the official CFWheels docs! The code and description above have been modified to use arguments scope.

Run your CFWheels database migrations automatically on application start

It’s no secret that I’m a massive fan of the Coldfusion on Wheels framework.. and there are a couple of tools that I use in my deployment process that are invaluable to me.

One of those is the DBMigrate plugin. I recently came up with a way to automatically migrate to the latest database version when the application first starts.

This can be useful/essential when:

  • Your deployment process doesn’t use post-deploy hooks
  • You want to make your application more portable
  • You want to simplify your deployment script

Essentially, it calls a couple of the plugin’s methods whilst doing a little array-fu.

/events/onapplicationstart.cfm

<cfscript>
// migrate database
_dbm = {}
// create a pointer to the dbmigrate plugin
_dbm.plugin = application.wheels.plugins.dbmigrate
// create an array of available migrations sorted by version in descending order
_dbm.available = []
for(item in _dbm.plugin.getAvailableMigrations()){
_dbm.available.Append(item.version)
}
ArraySort(_dbm.available, "numeric", "desc")
// migrate to the most recent version
_dbm.plugin.migrateTo(_dbm.available[1])
</cfscript>

There are a few caveats:

  • It will execute every time the application loads OR is manually reloaded via the reload=true parameter (though it’s fairy lightweight when there are no migrations to execute)
  • It will execute on every server in your cluster
  • There is (currently) no error handling
  • It could disable your application if your migrations fail