Install Python 3.4, Django 1.7 on Ubuntu 14.04 in a venv Virtual Environment

Python’s venv module provides support for creating lightweight “virtual environments” with their own site directories, isolated from system site directories. Each virtual environment has its own Python binary (allowing creation of environments with various Python versions) and can have its own independent set of installed Python packages in its site directories. – python.org

It’s well worth the trouble of setting up your Django projects this way, as system updates (eg: a system Python upgrade) won’t break your “virtual” projects. The venv concept has a big advantage over a traditional virtual machine, as there are no extra resources required to run another operating system. Win!

Run the following commands at the terminal to install Python 3.4 in a “virtual environment” and install Django inside the venv.

Install python3 and curl

sudo apt-get install python3 curl -y

Go to your home directory

cd ~

Create a virtual environment called “myvenv”

pyvenv-3.4 ~/myvenv --without-pip

Activate your new virtual environment

source ~/myvenv/bin/activate

Go to your new virtual environment directory

cd ~/myvenv

Create a directory and extract the python setup tools into it

mkdir pypioffline
cd pypioffline
curl -O https://pypi.python.org/packages/source/s/setuptools/setuptools-6.1.tar.gz
tar xvzf setuptools-6.1.tar.gz

Run the setup tools

cd setuptools-6.1
python ez_setup.py

Install pip

easy_install pip

Install django

pip install django

Run the Django web server

python manage.py runserver

Now visit http://127.0.0.1:8000/ to (hopefully) see your “Welcome to Django” page!

Other handy commands you’ll use…

Deactivate your new virtual environment

deactivate

Reactivate your new virtual environment

source ~/myvenv/bin/activate

Credits: Code Ghar

Passing a list of strings to CFWheels’ findAll() where argument

Use the ListQualify() function to pass a list of strings to the where argument in CFWheels ORM finders. ListQualify() encloses each list element in with the string specified (in this case, a single quote). Without it, CFWheels interprets the list as a single value.

<cfscript>
names = "Foo,Bar";
users = model("User").findAll(where="firstname IN ('#names#')");
/*
 * Generates SQL Query.. Not so good
SELECT firstname, lastname FROM users WHERE firstname IN ('Foo,Bar')
*/

users = model("User").findAll(where="firstname IN (#ListQualify(names, "'")#)");
/* 
 * Generates SQL Query. Great success!
SELECT firstname, lastname FROM users WHERE firstname IN ('Foo','Bar')
*/
</cfscript>

Great success!

Ensuring Full Unit Test Coverage In Your CFWheels App

Ensuring that you have unit tested all (or most) of your code in your CFWheels app is an important part of any test suite. As you hopefully know by now, the CFWheels framework features an in-built unit test framework.

The code below is a test itself, that will scan your controllers, models and views folders and make sure that you have a corresponding “test” cfc for each. If it discovers that a test is missing, it will fail.. so your app will not pass unless it has full test coverage.

To get it working, just copy the code below and save it into a file in /tests/TestsExist.cfc (or anywhere under the tests folder actually). You can retro-fit it to any app, though you may need to re-organise your test files somewhat.

There are some conventions that you need to follow to have it work properly, but I consider this being well organised (and you should be used to conventions by now). Your folder structure under /myapp/tests/ should look like this... Say you have a Login.cfc controller, your controller test cfcs themselves should be named like this TestLogin.cfc.. but you could also have multiple test files for your Login.cfc, just as long as the file BEGINS with “TestLogin”, eg: TestLoginRedirects.cfc, TestLoginBusinessLogic.cfc.. you get the idea.

The same goes for models and views. There is much one could do to improve/re-factor/customise this code, but it will hopefully catch anything you may have forgotten to test. Feel free to contribute to this project on GitHub.

So to summarise:

  • Re-organise your folders as per the example above
  • Re-name your test cfc files using Test{YourControllerName}.cfc
  • Copy the code into a TestsExist.cfc file under the /tests/ folder
  • Run your unit tests
<cfcomponent extends="wheelsMapping.Test">
	
	<cffunction name="setup">
		<cfset testsPath = ExpandPath("tests")>
		<cfset controllersPath = ExpandPath("controllers")>
		<cfset modelsPath = ExpandPath("models")>
	</cffunction>
	
	<cffunction name="test_00_setup_and_teardown">
		<cfset assert("true")>
	</cffunction>

	<cffunction name="test_01_all_controller_tests_exist">
		<cfdirectory action="list" directory="#controllersPath#" filter="*.cfc" name="controllers" />
		<cfdirectory action="list" directory="#testsPath & '/' & 'controllers'#" filter="*.cfc" name="controllerTests" />
		<cfset assert("controllerTests.recordCount gt 0")>
		<cfloop query="controllers">
			<cfif ListFindNoCase("Controller.cfc,Wheels.cfc", controllers.name) eq 0>
				<cfset abbrev = ListFirst(controllers.name, ".")>
				<cfset isTestFileFound = false>
				<cfloop query="controllerTests">
					<cfif FindNoCase("Test" & abbrev, controllerTests.name) gt 0>
						<cfset isTestFileFound = true>
						<cfbreak>
					</cfif>
				</cfloop>
				<cfset assert("isTestFileFound", "controllers.name")>
			</cfif>
		</cfloop>
	</cffunction>
	
	<cffunction name="test_02_all_model_tests_exist">
		<cfdirectory action="list" directory="#modelsPath#" filter="*.cfc" name="models" />
		<cfdirectory action="list" directory="#testsPath & '/' & 'models'#" filter="*.cfc" name="modelTests" />
		<cfset assert("modelTests.recordCount gt 0")>
		<cfloop query="models">
			<cfif ListFindNoCase("Model.cfc,Wheels.cfc", models.name) eq 0>
				<cfset abbrev = ListFirst(models.name, ".")>
				<cfset isTestFileFound = false>
				<cfloop query="modelTests">
					<cfif FindNoCase("Test" & abbrev, modelTests.name) gt 0>
						<cfset isTestFileFound = true>
						<cfbreak>
					</cfif>
				</cfloop>
				<cfset assert("isTestFileFound", "models.name")>
			</cfif>
		</cfloop>
	</cffunction>

	<cffunction name="test_03_all_view_tests_exist">
		<cfdirectory action="list" directory="#controllersPath#" filter="*.cfc" name="controllers" />
		<cfdirectory action="list" directory="#testsPath & '/' & 'views'#" filter="*.cfm" name="viewTests" />
		<cfset assert("viewTests.recordCount gt 0")>
		<cfloop query="controllers">
			<cfif ListFindNoCase("Controller.cfc,Wheels.cfc", controllers.name) eq 0>
				<cfset abbrev = ListFirst(controllers.name, ".")>
				<cfset isTestFileFound = false>
				<cfloop query="viewTests">
					<cfif FindNoCase("Test" & abbrev, viewTests.name) gt 0>
						<cfset isTestFileFound = true>
						<cfbreak>
					</cfif>
				</cfloop>
				<cfset assert("isTestFileFound", "controllers.name")>
			</cfif>
		</cfloop>
	</cffunction>

	<cffunction name="teardown">
	</cffunction>

</cfcomponent>

Automated Testing with CFWheels, Jenkins & Ant Tutorial

Automated Testing / Continuous Integration sounds complex and intimidating, but its like brain surgery.. its not that hard once you know the concepts and how the different elements fit together. Well maybe brain surgery wasn’t such a good example.. but you get the idea. Just think of a CI server as a fancy scheduled task manager.. with a few extra bits & pieces bolted on.

One of my pet projects this year has been to completely automate the testing and build of a CFWheels application. This guide is a bare-bones, no-frills installation to get you up and running. Assuming you already have written your unit tests, you should be able to get this up and running in under an hour.

There are many ways to skin this cat, but for this example I have used Jenkins as the CI server, Git as the version control system and Ant to do the builds. All elements are open source, so you can get up and running for zero dollars. I’m sure there are arguments for or against these choices and it’s quite possible to substitute any of them with your preferred method. There are an extensive array of plugins available for different source control flavours and build ‘tools’ (Ant, Maven, Command Line). My installation is on a 64bit version of Windows 7 using default installation paths.

Ideally, you would have a dedicated CI server which has its own application server (ColdFusion/Railo) and database. This is not essential.. in fact, my working example was installed on my laptop.. the only drawback with having the installation on your development machine, it that you will have to deploy your app to your webroot using a different foldername. Eg: “{webroot}/wheelsapp” source code will have to be deployed to “{webroot}/wheelsorama” so it can be tested without affecting your source code.

If you wanted to make your configuration even simpler, you could remove the Git repository configuration, and just run the unit tests directly on your source code.. but this is not ideal. * Be VERY careful with taking this approach as the example build script deletes the destination app folder before each deployment.. and that would be bad mmkay.

The example below will enable you to test your application, your plugins, the core wheels framework or a combination of the three (see the build.xml file).

Software Installation

As a general rule, I have install all software in their default locations, as you will see in the Jenkins configuration steps.

1. Install Java JDK
2. Install Ant (I recommend Winant if you’re on Windows)
3. Install Git
4. Install Jenkins

Windows users may need to re-boot (surprise surprise!) if the Jenkins service is not running (Check your Windows Services to confirm), if the service still fails to start, you may need to execute this from a command prompt (Run as administrator) An inconvenient workaround!

cd C:\Program Files (x86)\Jenkins
java -jar jenkins.war --ajp13Port=8010

5 .Visit your Jenkins server at http://localhost:8080 (or port 8010 if you had to start Jenkins via the command prompt)

You should have a brand-spanking new Jenkins server with which to unleash your automated wrath!

Configure Jenkins

1. Click “Manage Jenkins” > “Manage Plugins” > “Available” Tab
2. Find the Git Plugin and install it
3. Click the “Installed” tab and ensure you have the Git and Ant plugins. (Ant plugin should be bundled with the Jenkins install)

4. Return to the Jenkins dashboard and click “Manage Jenkins” > “Configure System”
5. In the JDK section, uncheck the “Install automatically” option and enter a name and the path to your JDK folder.

6. In the Git section, uncheck the “Install automatically” option and enter the path to your git executable

7. In the Ant section, click the “Add Ant” button, uncheck the “Install automatically” option and enter a name and the path to your Ant folder (or WinAnt if installed).

Create your Job

1. Return to the Jenkins dashboard and click “New Job”
2. Enter a job name and select “Build a free-style software project”

3. Under the “Source Code Management” section, check the “Git” option
4. Specify your app’s Git repository. If you are unfamiliar with this, it’s time to click here. This can be a local path, a network path or an ssh address.

5. Under the “Build Triggers” section, choose your preferred build interval. I like to have Jenkins poll my SCM (Git repository)  for changes every few minutes and build automatically if it finds any commits since the last build. */5 * * * * means every 5 minutes.. Click on the help icon for more info about intervals.

6. Under the build section, click the “Add Build Step” button and select the “Invoke Ant” option. Select the “Default Ant” configuration you added earlier. Since we will be using the default Ant convention (using build.xml), this is all that’s required.. but more about that shortly.

7. In the Post-build action section, click the “Add post-build action” button and select the “Publish JUnit test result report” option. Specify where your unit test result xml files will be written to. In this case build-report/test_result_*.xml, a folder called build-report with the files being called test_result_{something}.xml. Don’t worry too much about any warning messages here.. as the folders don’t exist yet.

Prepare your Wheels app 
There are a few steps here..

  • Initialise your wheels app as a Git repository
  • Download the JUnify Wheels plugin (the most dangerous plugin in the galaxy) which will return your unit tests in JUnit format (Ties in very nicely with Jenkins)
  • (Optional) Download the DBMigrate plugin for making automated database changes during the build
  • Now we need to tell Ant what to do to build and test your app. We will do this by creating 2 files in your app’s root folder. The 2 files are ant.properties and build.xml. ant.properties is for setting variables for use in build.xml. build.xml tells Ant what to do. * You will most likely need to modify the ant.properties file to suit your environment

ant.properties

# for running start/stop bat files
railo_bin=C:/railo/tomcat/bin
# cf_bin=C:/ColdFusion9/bin
# your webroot.. where jenkins will deploy your code (Apache/IIS)
web_root_dir=C:/railo/tomcat/webapps/ROOT
# web_root_dir=C:/inetpub/wwwroot
# where jenkins will git pull your code.. then deploy from
jenkins_workspace_dir=C:/Program Files (x86)/Jenkins/jobs/Wheels-O-Rama Deploy/workspace
# where jenkins will deploy to
destination_app_dir=${web_root_dir}/wheelsorama
# where unit test results will be written to
build_report_dir=${jenkins_workspace_dir}/build-report
# your app's url
app_root_url=http://localhost:8888/wheelsorama/index.cfm
# the unit test base url
unit_test_url=${app_root_url}?controller=JUnify&amp;action=index&amp;

build.xml

<project default="initialize">

	<!-- this is the default target (function) -->
	<target name="initialize">

		<!-- import properties -->
		<property file="ant.properties" />

		<!-- stop railo service.. not always needed! -->
		<!-- <exec dir="${railo_bin}" executable="cmd" os="Windows NT">
			<arg line="shutdown.bat"/>
		</exec> -->

		<!-- once jenkins has pulled the code into its workspace, i want to delete the previous deployment, but keeping root folder -->
		<mkdir dir="${destination_app_dir}" />
		<delete includeemptydirs="true">
			<fileset dir="${destination_app_dir}" includes="**/*" />
		</delete>

		<!-- then get a fresh copy of the latest code into the webroot for testing -->
		<copy todir="${destination_app_dir}">
	    	<fileset dir="${jenkins_workspace_dir}"/>
	    	<!-- exclude any files you don't want/need -->
	    	<!-- <exclude name="**/.gitignore"/> -->
	    	<!-- <exclude name="**/*.php"/> -->
	    	<!-- <exclude name="**/*.asp"/> -->
		</copy>

		<!-- make a folder in the workspace to keep my build reports -->
		<mkdir dir="${build_report_dir}" />

		<!-- start railo service -->
		<!-- <exec dir="${railo_bin}" executable="cmd" os="Windows NT">
			<arg line="startup.bat"/>
		</exec> -->

		<!-- migrate database using a call to the dbmigrate plugin url -->
		<!-- <get src="${app_root_url}?controller=wheels&amp;action=wheels&amp;view=plugins&amp;name=dbmigrate&amp;password=yourpassword&amp;migrateToVersion=999999999" dest="${build_report_dir}\dbmigrate_result.html" /> -->

		<!-- make a get request to your unit test url, writing the result to a file. Jenkins will use it to determine the success of your build
			== possible type param values ==
			# type=app		: runs the tests in /myapp/tests
			# type={myplugin}	: runs the tests in /myapp/plugins/{myplugin}/tests
			# type=core		: runs the tests in /myapp/wheels/tests
		-->
		<get src="${unit_test_url}type=app" dest="${build_report_dir}\test_result_app.xml" />
		<!-- <get src="${unit_test_url}type=myplugin" dest="${build_report_dir}\test_result_myplugin.xml" /> -->
		<!-- <get src="${unit_test_url}type=core" dest="${build_report_dir}\test_result_core.xml" /> -->

	</target>

</project>

Unit Tests

This is the final piece of the puzzle. The Wheels team have done a great job documenting unit testing your app using the Wheels test framework, and there is a post by someone about unit testing your plugins.. So get on it.

Build your app

Now that you are all configured and unit tests written.. its time to build! Click on the “Wheels-O-Rama Deploy” job then in the left menu click “Build Now”. You should see a progress bar appear under the menu.. it should either return a blue (good) or red (bad) icon. If you mouseover the build link and click the “Console Output” link you will see the output that was produced by Jenkins & Ant. You can watch the console output in real time by clicking the link whilst the build is in progress. If the build fails, this is where you will find the clues to its resolution.

Umm.. so WTF is actually happening?

I’m glad you asked..

  • Jenkins polls your Git repository looking for commits
  • When a commit it found, it will pull your source code into its workspace
  • Jenkins will use Ant to run your build.xml file which will…
    • Delete the previous code deployment from your webroot
    • Copy the latest source code from Jenkins workspace into your webroot
    • Call your unit tests via a URL and write the results to the Jenkins workspace
  • Jenkins will analyse the unit test results
  • Jenkins will report whether the build passed or failed

Where to from here?

This post is really just a minimal working example of automated testing. Here are a few ideas to play around with:

  • Setup a post build action to alert you of build failures
  • Break the existing job into separate “Deploy” and “Test” jobs (using a “Post Build Action” to kick off the Test job)
  • Build and test with different CFML engines, CFWheels framework versions and database engines for wide compatibility
  • Use the dbmigrate plugin to automatically make database changes upon building (See build.xml example)
  • Create another job that uses Ant to FTP your successful build to your production server
  • Use Selenium to do automated functional browser testing
  • Push your successful build to GitHub
  • Extending Jenkins further with the smorgasbord of plugins. The Chuck Norris plugin is a favourite of mine!

Feel free to comment regarding your experiences using this guide.. Thanks for reading!

JUnify Plugin for CFWheels

Make the CFWheels test framework return jUnit XML

JUnify will run your CFWheels unit tests, then return the results in jUnit formatted XML. Use it for automated testing with your Continuous Integration server.

Following the road to jUnit Glory

  • Download and install the JUnify plugin from CFWheels or GitHub
  • Create a JUnifyController.cfc controller that calls the JUnify() function*
  • Write some unit tests for your app or plugin
  • Make a call to your JUnify url

* Controller filename and action names are unimportant

JUnifyController.cfc

<cfcomponent extends="Controller">
	
	<cffunction name="index">
		<cfset JUnify(params)>
	</cffunction>
		
</cfcomponent>

Your JUnify URL

http://yourserver/index.cfm?controller=junifycontroller&action=index&type=app

Pay particular attention to the type parameter.
– type=app will run tests for your application
– type=myplugin will run tests for your plugin
– type=core will run tests for the CFWheels framework itself

Props to Rocketboots for their RocketUnit test framework

DatePicker Plugin for CFWheels

Use a jQuery UI datePicker widget in your CFWheels forms

datePicker and datePickerTag functions accept the same arguments as CFWheels textField and textFieldTag functions respectively, but they also accept all options for the jQuery datePicker widget

Following the road to DatePicker Glory

1. Install the DatePicker plugin from CFWheels or GitHub
2. Make sure you have setup jQuery AND jQuery UI in your app (consider my JQuack plugin)
3. Use the datePicker() and datePickerTag() form helpers

Examples

<!--- Use with objects --->
#datePicker(label="Birthday", objectName="user", property="birthday")#

<!--- Use without objects --->
#datePickerTag(label="Birthday", name="birthday", value=Now())#

Note: All jQuery specific arguments (jQuery datePicker options) are case-sensitive

CFWheels Plugin Function Argument Defaults

A handy and powerful CFWheels feature for keeping your code DRY is to be able to configure application-wide default for wheels functions.

It’s also possible to tap into the CFWheels framework to make this feature available to functions added by your own plugins. Here’s how.. (props to Andy Bellenie)

Let’s assume you have created a plugin called putOnWeight and in this plugin, you have a function called eatCake()

As per the CFWheels documentation you set your application-wide function defaults in your settings.cfm like so:

<!--- config/settings.cfm --->
<cfset set(functionName="eatCake", howMany=20)>

To have your plugin accept these function defaults, you will need to tap into the CFWheels $args() function.. like so:

<!--- plugins/PutOnWeight.cfc --->
<cfcomponent output="false" mixin="controller">

	<cffunction name="init">
		<cfset this.version = "1.1.7">
		<cfreturn this>
	</cffunction>

	<cffunction name="eatCake">
		<!--- note that howMany argument is not required, and has no default --->
		<cfargument name="howMany" type="numeric" required="false">
		<!--- use the CFWheels $args function to look for default arguments set in settings.cfm --->
		<cfset $args(name="eatCake", args=arguments)>
		<!--- cake.. nom nom nom! --->
		<cfset var loc = {}>
		<cfset loc.cakesEaten = arguments.howMany>
		<cfreturn loc.cakesEaten>
	</cffunction>

</cfcomponent>

Disclaimer: the $args() function is a private cfwheels framework function so use at your own peril… I am.