Skip to content

Build script

Jump to bottom Edit New page
theopak edited this page Aug 6, 2011 · 88 revisions

What it does

  • Combines and minifies javascript (via yui compressor)
  • Combines and minifies CSS
  • Optimizes JPGs and PNGs (with jpegtran & optipng)
  • Removes development only code (any remaining console.log files, profiling, test suite)
  • Basic to aggressive html minification (via htmlcompressor)
  • Revises the file names of your assets so that you can use heavy caching (1 year expires).
  • Upgrades the .htaccess to use heavier caching
  • Updates your HTML to reference these new hyper-optimized CSS + JS files
  • Updates your HTML to use the minified jQuery instead of the development version
  • Remove unneeded references from HTML (like a root folder favicon)
  • Runs your JavaScript through a code quality tool (optional)

How to Use

If you're on Mac or Linux...

You've got all your dependencies pre-installed, likely. You may need a yum install ant-contrib or what have you.

If you're on Windows...

  • Get the Java JRE (or JDK)
  • Get WinAnt and point the installer to Program Files/Java/jre6/bin/
  • Then open up a command line interface by doing Start Menu > Run > cmd.exe.
    • And of course, navigate to your project folder.
  • However, We created a friendly runbuildscript.bat file for you windows users if you'd like to avoid the command line.

Navigate into the build folder:

cd build

Build your site:

ant build

There are a few different build options:

ant build     # minor html optimizations (extra quotes removed). inline script/style minified (default)
ant buildkit  # all html whitespace retained. inline script/style minified 
ant basics    # same as build minus the basic html minfication
ant minify    # same as build plus full html minification
ant text      # same as build but without image (png/jpg) optimizing

Your build will be added to the publish/ folder. BOOM! you're done.

Configuration

You can override what folders and files you want to operate on in project.properties. All the default configuration is in default.properties.

You might also need to dig into the build.xml to make stuff happen. It might look a little scary, but, trust us, it used to be much scarier! ;)

You can now add multiple stylesheets to the project (defined in the project.properties file) and they will be concatenated into 1 file, with any default stylesheets defined in the default.properties file being added at the end.

Adding new 'pages' (html, php, etc) is handled in the same way and the script should iterate through any files listed here and update the css / javascript references as per the original script

We have also added the ability to define 'environments' for the build process. The original targets still exist, and run the same functions, however each target now has a prod, test and dev environment too.

Here is how the environments work:

  • dev - Increases build number, cleans and copies the build and optimises any images if the target originally did
  • test - Runs everything that the original target did, however it does not strip the console.log or profiling parts
  • prod (default) - Runs everything the original target did

To run it you simply use ant <target> -Denv=<environment>

ant build -Denv=dev

If you don't provide the env variable it will use the production environment

Configuration Cookbook

The following outlines some basic (and not so basic) tasks that you might need to accomplish when using the build script.

As always, feel free to pitch in and add your own examples.

Wordpress integration

Check out Jay George's wordpress screencast videos: http://www.jaygeorge.co.uk/1-html5-buildscript-wordpress-1/

Using the build script on a Wordpress theme introduces a significant problem: Wordpress themes are installed to a website's /wp-content/themes/<theme-name> directory while the build script's output references the minified/concatenated files in a relative URI as js/scripts-xxxx.min.js. Therefore, output of the build script will reference files in the wrong location.

To make the build script output files that reference the correct url /wp-content/themes/<theme-name>/js/scripts-xxxx.js, the build.xml file must be modified. Edit the 2 regular expressions near lines 599 (for js) and 699 (for css) to include the Wordpress PHP that dynamically references the theme's folder: (the line to add is indented)

<replaceregexp match="&lt;!-- scripts concatenated [\d\w\s\W]*?!-- 
end ((scripts)|(concatenated and minified scripts))--&gt;" 
replace="&lt;script src='
    &lt;?php bloginfo('template_url'); ?&gt;/
${dir.js}/scripts-${build.number}.min.js\'&gt;&lt;/script&gt;" flags="m">

Basic Tasks

Almost every project that uses the build script will need to add/delete or move files and folders from the build script. In the simplest cases this can be achieved by editing the project.properties file included in build/config/

Moving and renaming folders

For something as simple as changing the name of the CSS directory from "css" to "styles" add the following to the "Directory Structure" section

 # Directory Structure
 #
 # Override any directory paths specific to this project
 #
 dir.css	= styles

Some folks like to park static assets in a separate folder to segregate them out from the rest of the application code. If, for example, you wanted to move all of your static files to a folder named "_assets", your Directory Structure section would look something like this:

 # Directory Structure
 #
 # Override any directory paths specific to this project
 #
 dir.js = _assets/js
 dir.css = _assets/css
 dir.images = _assets/images

configure deep nested files

There is a special syntax one could use to include or exclude deep nested files in config.

For example, if you put compass source files inside css/src/ folder. but if you are using a setting like the one below, the build script still includes scss files in the build result.

file.exclude = *.scss

To solve this issue, you could use the syntax below:

file.exclude = **/*.scss

Running JSHint and JSLint

Optionally, you can have the build script run your JavaScript files through JSHint or JSLint to help detect errors or potential problems. This is not necessary for any of the other build options, and build failure when running these targets only means that errors were found during the quality check. To run, simply specify one of the following:

ant build jshint
ant build jslint

JSHint and JSLint can both be configured with a variety of options. The defaults can be found in the default.properties file, and you can override them in project.properties. See http://jshint.com/ and http://www.jslint.com/lint.html for more information.

Note that these targets are set to exclude any JavaScript files ending in ".min.js", as well as files located within the js/libs directory.

Running CSSLint

You can check if your CSS files have any issues by running your files through CSSLint and detecting potential problems (note that CSSLint parser is not perfect and has bugs). Build failure when running these targets only means that errors were found during the quality check of your stylesheet. To run, use:

ant build csslint

Advanced Tasks

The following require a bit more work, often touching on editing or creating tasks in the build.xml itself

Controlling the order of JavaScript

If you want to concatenate several JavaScript files and the order is important then you can replace the wildcard based concat with specific file names. It's more work, but hopefully you won't have to tweak this too often.

For example, the JavaScript task (around line 445) will go from:

    <target name="-js.main.concat" depends="-load-build-info" description="(PRIVATE) Concatenates the JS files in dir.js">
        <echo message="Concatenating Main JS scripts..."/>
        <concat destfile="./${dir.publish}/${dir.js}/scripts-${build.number}.js">
            <fileset dir="./${dir.publish}/">
                <include name="**/${dir.js.main}/*.min.js"/>
                <exclude name="**/${dir.js.mylibs}/*.js"/>
                <exclude name="**/${dir.js.libs}/*.js"/>
            </fileset>      
        </concat>
    </target>

to this:

    <!-- JAVASCRIPT -->
    <target name="-js.main.concat" depends="-load-build-info" description="(PRIVATE) Concatenates the JS files in dir.js">
        <echo message="Concatenating Main JS scripts..."/>
        <concat destfile="./${dir.publish}/${dir.js}/scripts-${build.number}.js">
<!--Add one entry for every file-->
	        <fileset file="./${dir.publish}/${dir.js}/plugins.js" />
      		<fileset file="./${dir.publish}/${dir.js}/script.js" />
        </concat>
    </target>

Some notes

  • The build script makes some assumptions about how your javascript files are sorted.
    • /js/libs/ contains files from boilerplate. These are minified but not concatenated with others. Modernizr should be alone in the head. jQuery might be pulled off the CDN, and the pngfix is IE6 only.
    • /js/mylibs/ has your other javascript libraries and plugins.. All files in here will be minified (unless they end with .min.js) and then concatenated together.
    • plugins.js and script.js in the /js/ folder are all yours. these will also be min'd and concat'd (after) with the mylibs files.
  • When the build script changes your HTML to reference the new min'd script (usually named something like scripts-002.min.js) it looks for some HTML comments which refer to the beginning and end of the script block. Currently it looks for <!-- scripts concatenated and <!-- end scripts-->. If you change or strip these comments, the build script will kinda die. :)
  • It likes your filename extensions to be lowercase. Some things might be skipped if you use a .JPG extension, for example.
  • Right now, the build.xml et all all have to be in the ./build/ folder. You can't rename it.
  • so all of mylibs min files go into /libs-xxx.min.js
  • all scripts/*.min.js go into scripts-xxx.min.js
  • and then those get smushed together into scripts-xxx.min.js again.
  • Check out issue #165 for everything that's planned for next version
  • Although not officially supported, it is possible to set the publish directory to a relative path outside the project folder. This is useful for generating a snapshot onto a staging server, for instance. In order to do so, set the publish directory to a path relative to the project: dir.publish = ../mysite/. Then tell the htmlcompressor script to put the files back where it found them, instead of defining a specific directory. In build.xml, this line <mapper type="glob" from="*" to="../${dir.publish}/*"/> becomes this line <mapper type="glob" from="*" to="*"/>.
  • Intermediate stages are stored in a new intermediate folder, and only files that should be published are copied into the publish folder.
  • Files are not deleted at the beginning of every build, and files that have already been processed will not be reprocessed unless the source has changed.
  • Versioned files are referenced by a SHA-1 hash of the content rather than a build number. This means that changing your HTML and rebuilding will not cause your users to redownload the same CSS and Javascript, and a reverted change may cause users to use a copy that was previously downloaded. It may be better to use only part of the hash so the HTTP request is shorter.

Platforms

It should work fine on Windows, Mac and Linux. We are committed to this working cross-platform.

Dependencies

On Linux/Mac, the image tasks assume you have jpegtran (part of the libjpeg-progs package http://www.ijg.org/) and optipng (http://optipng.sourceforge.net/) installed and on your path. The binaries are included for Windows users.

On Mac OSX

Using homebrew, install the following packages:

brew install libjpeg optipng

Using MacPorts, install the following packages:

port install jpeg optipng

On Ubuntu

Using apt, install the following packages:

apt-get install libjpeg-progs optipng

Easy VCS-based deployment

Check out the entire repo incl the build/ folder onto your server.

Then go into build/ and run ant.

Then drop this into the top of your .htaccess:

RewriteEngine On
RewriteCond $1 !^yourapp/publish/
RewriteRule ^(.*)$ yourapp/publish/$1 [L]
# and change yourapp to your actual checkout folder name

This allows you to serve a subfolder as if its the root of your site. So now to update production you can just to do a git pull && cd build && ant build or what have you, whenever you want it updated.

Troubleshooting

If you are using Ubuntu Linux and receiving the following error when running ant:

Unable to locate tools.jar. Expected to find it in /usr/lib/jvm/java-6-openjdk/lib/tools.jar

See this blog post for how to resolve the problem.

If you receive the following errors:

  BUILD FAILED
  build.xml:136: The following error occurred while executing this line:
  build.xml:441: concat doesn't support the "overwrite" attribute

This is because versions of Ant prior to 1.8.2 do not support the overwrite="no" attribute.

You will need to either upgrade to a later version of Ant or remove all of the occurrences of overwrite="no" from the build.xml file.

Ports

There is an effort underway to port things to rake. Woo!

Add a custom footer
Add a custom sidebar
Clone this wiki locally