jump to navigation

Don’t use NPM install -g inside Vagrant October 20, 2015

Posted by willhlaw in Development.
Tags: , , ,
add a comment

Using Vagrant for your node.js projects makes sense if you want an environment that can be reliably and repeatedly built by any one of your team members and new ones without going through the hassle of writing Getting Started instructions for getting elasticsearch installed for Windows, Mac, and other *nix. I highly recommend it…or it’s up and coming predecessor, Otto (which actually uses Vagrant under the hood, but hides a lot of the complexity and uses best practices).

However, you cannot have the same start up process each and every time if you install dependencies without pinning or locking in the specific version. So rather than running:

>npm install -g grunt-cli@0.1.13

Instead, install grunt-cli locally as a dev-dependency and the `grunt` command will be available as npm scripts in your package.json’s script {} block.

>npm install grunt-cli@01.1.13 save-dep
>npm shrinkwrap --dev

We capture the version npm grabbed and stored in package.json under the devDependencies {} block by running `npm shrinkwrap` with the ‘dev’ flag. This locks your entire (local) dependency tree from your package.json, and creates an npm-shrinkwrap.json file. Npm-shrinkwrap.json files are not used for global installs. So next time your team member starts work on your project and runs `vagrant up` and then `npm install`, the locked dependencies will be read from npm-shrinkwrap.json (instead ofpackage.json) and your team member’s node_modules/ directory will behave just like yours.

npm-shrinkwrap.json files are not used for global installs — npm issues

So how then exactly do we run `grunt` from the command line since we don’t install it globally?

In package.json, we add some scripts:

  "name": "myProject",
  "version": "0.0.1",
  "devDependency": {
    "grunt-cli": "^0.1.13"
  "scripts": {
    "grunt": "grunt",
    "test": "grunt test"

And from the command line or another script, you run it with:

>npm run test  ## executes `grunt test`
>npm run grunt -- --help  ## executes `grunt --help`

The two extra dashes is npm’s way of allowing you to pass any grunt parameters after it.

To conclude, use npm-shrinkwrap within Vagrant for repeatable and reliable npm environment for your node.js project and install global packages locally and call them using `npm run <scripts.name>`.


Mobile App and Website Testing Roundup for 2015 April 5, 2015

Posted by willhlaw in Development, Mobile, Testing.
Tags: , , , , , , ,

After researching testing for a startup, I thought I would share my findings.

Background on Testing

There are a few concerns that testing attempts to solve that is applicable to most mobile app product deployments.

  • There is functional testing (is anything broken?).
  • There is load testing (does app or website fold under pressure?).
  • And there is usability testing (do paid testers acting like users find the app or website easy to use?).
  • Then, there are the targets. Need to consider performing these tests against all of the combination of workflows, iOS and Android and the backend server and database as well as any websites.
  • iOS and Android app testing is different, see why
  • List of Testing Tools for mobile and others

Appium (site, review) – Free, required for services like AppThwack

Free, open source, raw foundation to create test scripts that can be written outside of the project’s code base. Has improved much over the past two years to become the tool of choice among QA shops. I chose this over other testing frameworks such as MonkeyTalk and  Robotium.

  • Free
  • Requires coding knowledge and uses black box testing Selenium style
  • It would be ideal if the development team uses this framework or one of the aforementioned, because other testing websites like AppThwack expect to run these test scripts.

AppThwack (site, review) – Cheap, requires test scripts like Appium

If you have test scripts running for your project, like Appium or others, then AppThwack can automate the running of those tests on 100s of real devices.

  • $20/month for 200 device test minutes up to $500/year for 7,500 device minutes
  • Requires suite of test scripts to be already written

TestElf (site, review) – Cheap

Covers functional testing and has 2 day turn-around

  • $50 signup offer, $200 for 1 test, $1,000 for 6 tests, $2,000 per month

UserTesting (site, review) – Variety

Shows videos of test users using the app or website

  • $49/video or $3,000/year

Offers a free version called Peek, but the app needs to be in the app store.

  • Free

Applause (site, review) – Expensive

Following estimates (see and modify actual quote) are for 4 Apps (iOS App and Android count separately, and there is Consumer and Merchant app):

  • 4 Apps Functional Testing costs $4,500 – $7,499*
    • *With annual subscription, and salesman said these autoquote numbers are high and can come down
    • Custom team of testers and allows up to 10 test case hours
  • 4 Apps Load Testing costs $18,000 – $30,000*
    • *Salesman said these autoquote numbers are high and can come down
    • Led by a performance engineer and hand creates tests and will create reports and improvement recommendations


  • Try TestElf’s functional testing by paying $50 special offer
  • Try Peek’s usability testing for free
  • Depending on seriousness and budget, start to negotiate with Applause and/or require development team to create a suite of tests to cover each major requirement and workflow