'Tech' blog posts

Integrating Karma code coverage with TeamCity

To unit test our Angular apps we use Karma test runner and Jasmine testing framework. Locally we run these tests using a gulp script that takes care of the whole app building process. To ensure nothing is broken before publishing the app to production we run our tests during the continuous integration process using TeamCity.

This post expects you to have a gulp testing process already in place and it won't cover that part. It also expects you to have a working TeamCity setup in place. The post will only help you integrate Karma with TeamCity as an additional build step so you would get something that looks like this in your TeamCity.

Number of passed/failed tests:

The code coverage tab:

There are a few requirements before we can make this work. To help you better understand our setup, here is a sample project structure that we have:

The first thing to do is ensure you have the following npm packages installed and that they are saved in your package.json file:

"karma": "^0.13.22",
"karma-chrome-launcher": "^1.0.1",
"karma-coverage": "^1.1.1",
"karma-jasmine": "^1.0.2",
"karma-phantomjs-launcher": "^1.0.0",
"karma-teamcity-reporter": "^1.0.0",

Next ensure that you have the following set up in your karma.conf.js:

  • "coverage" and "teamcity" in the reporters list
  • "PhantomJS" in your browsers list
  • singleRun set to true
  • our coverageReporter configuration looks like this (this part is pretty important):
coverageReporter: {
  dir: 'coverage',
  reporters: [
    { type: 'html', subdir: 'html' }
  ]
}
  • set the preprocessors configuration to something like this:
'path/to/code/you/want/to/tests/*': ["coverage"]
  • NOTE: we do not have the plugins property set up
  • the rest of options are pretty much standard - add/remove what you need

Now that this is all set up, go to your TeamCity. This is essentially how our client-side build process looks like:

The step that is the main interest of this post is the "Run Karma Tests" step. Here is how we have it set up (create a Command Line step):

This is a slightly modified version of what Karma documentation recommends. The difference is that we are forcing the use of local Karma module and we specify the configuration as a command line param like this:

node node_modules/karma/bin/karma start karma.conf.js

The last piece of the puzzle is setting up the coverage artifact. Go to the General Configuration Settings of your project in TeamCity and add an additional coverage artifact path (the second line):

The important bit (it's simply where our coverage html files are located):

Project.WebApp/coverage/html/** => coverage.zip

Go back and see how we have the coverage/html folder in our project structure. It is set up by coverageReporter karma.conf.js property. This artifact path will take all the files from the coverage/html folder and will compress it into a coverage.zip archive. After the build process finishes, TeamCity will (if it's is able to find the coverage.zip archive inside the artifacts folder) automatically import it as code coverage for the project and you will be able to navigate to the "Code Coverage" tab for that specific build. If you have any tests that don't pass, this will also fail the whole step, stop the build and prevent it from ending up in production.

Hope this helps. Cheers! :)


Visual studio 2015 real-time CSS editing

I was working on some updates to MenuCal this week. This morning while completing some CSS styling on a new form, I discovered that the CSS was being updated in real-time in Chrome as I made changes in visual studio. This is a huge improvement to my workflow, as I like to style and preview as I go. I was able to drag my CSS editor over to another screen and work away while the styles in Chrome updated instantly. No more hitting save, and refreshing the browser. Thank you Visual studio 2015!! I've seen other tools do this for quite some time, but it's nice to see it in the IDE I use every day.

But let's not get too excited. I mean who uses plain old CSS anymore? We've been using SASS on new projects and unfortunately this lovely little feature is not present out of the box for SASS. I will take a look around at VS plugins that might do that, and report back if I find some elegant solutions.

Update 2nd August, 2016: I tested out one of our projects with SCSS and Sassy Studio. While it's not as elegant as the live CSS preview, it does detect the CSS changes after they are compiled, and the browser updates the CSS.


This week in tech

I'm jotting down some notable tech news we've been discussing internally (in our slack #techtalk channel) this week.

We use New Relic on a number of applications, it's a great tool for highlighting performance issues in applications. Microsoft has always been somewhat in that game, but their new offering built into Azure is called "Application Insights". It looks to be a direct competitor to New Relic. It also has logging and a query engine to go with it, so it may also be aiming for cloud logging providers too (like Log Entries). https://azure.microsoft.com/en-us/documentation/articles/app-insights-overview/.

Trevor uses a mac (boo!), and we're a Microsoft development house. At times he struggles to find the right tools to work in a primarily windows environment, and he usually resorts to a virtual machine or RDP. We recently found this tool called Wagon (https://www.wagonhq.com/), and Trevor has been using it and enjoying it. Wagon is built on Electron, another tool we have been keeping an eye on lately. Fabrizio is especially enamored by it.

Apparently we care about API versioning. I'm not sure, but other people care about it more than me: Your API versioning is wrong, which is why I decided to do it 3 different wrong ways.

VHS won! But only barely. https://www.theguardian.com/technology/2015/nov/10/betamax-dead-long-live-vhs-sony-end-produtionhttp://news.sky.com/story/remember-vcrs-production-to-end-as-sales-slump-10509632.

Javascript jokes are so hot right now:

Lastly, John found this. Have we gone too far?


Team City - Update Packages

Here at Dovetail we love Team City and Visual Studio.

We recently updated our Team City configuration to allow projects to be built using Visual Studio 2015, C# 6, and to use the latest Nuget package manager.

In doing so, we discovered a very peculiar setting deep within Team City that caused one of our projects to break on build and break once deployed.

The Build Failures

After updating, we ran our build and the compiler threw an error saying that it could not find a specific version of a Nuget package. For example, our packages.config within Visual Studio specified we use Nuget to install Newtonsoft.Json version 7. However Team City reported that the project needed Newtonsoft.Json version 8.

We made the decision to update all affected nuget packages to the latest versions, pushed our project and Team City built it successfully.

The Deploy Failure

We then ran into our next problem. The project was deployed but there was nothing on the screen. We opened up Chrome developer tools and found that JQuery was missing. This is a project that uses a lot of JavaScript files and it built and deployed with no problems before.

Looking back at our Octopus Deploy package we found that the JQuery file we were referencing and pushing to our repository was not there any more. However, we did see the latest version of the JQuery min file. Our file was being removed and replaced with the latest JQuery min version.

The Update Package Setting

We soon found the setting buried deep inside the Team City "build steps" screens:

 undefined

Within the NuGet Installer build step is a setting which, when turned on, updates all your packages. This sounds great in theory but when you run into build and deploy issues this will cause headaches.

undefined

 

The text underneath states "Uses the NuGet update command to update all packages under solution. Package versions and constraints are taken from packages.config files". Whether this is a bug in Team City or not, this text seems very vague for an "Update Packages" function.

Be careful, because when checking this check-box, Team City will not read the packages.config version numbers and instead it will download the latest version of every package.

Update: Team City have been back to us and they're going to update the explanatory text on this checkbox to make it more clear.

 


Migrating from InfluxDb v0.8.7 to v0.9.6

undefined

In one of the applications we're working on we recently had to make a move from InfluxDb v0.8.7 to v0.9.6. Because the official migration paths didn't work for us (DB upgrades would either lose data or not finish at all) we had to develop a small c#/.net app that would reliably execute the migration for us.

We successfully migrated around 4GB of data with it and are quite happy with how it went. It did take quite a bit of time but all the data is safe and usable.

The app also lets you specify backfills (rollups) to be created once all the base data is migrated. 

Today we're open-sourcing this migration tool in hope it might help someone else make the move as well. :)


Open-sourcing InfluxData.Net library

We've been using the InfluxDb time-series database for almost a year now on one of our projects and it works pretty nicely even though it still didn't hit the v1.0 mark. 

We started our InfluxDb journey with v0.8.7 and thus far even though we wanted to, there wasn't an easy way to migrate to v0.9.x. We however came to a point where we needed to upgrade in order to implement new features required by the project.

The first step to take was to see if there are any .Net libraries that supported InfluxDb v0.9 and the one we've been using from start seemed to be the best one. The problem was it wasn't updated for quite a while and it didn't support the latest InfluxDb versions.

So, I decided to fork it, refactor it and make it work with the latest InfluxDb. The code can be found on GitHub, it's under MIT licence and there is also a NuGet package on nuget.org. The integration tests are all working again and the docs have been updated. Rejoice!

In the future, my plan for the library is to support the rest of the TICK stack layers as well as their API's get more stable.

We're also planning on open-sourcing the migration tool that we developed and used to migrate the data from v0.8 to v0.9 in hope it might help someone else as well. :)


Failing Azure Recovery Service VM restore jobs and how to resurrect your backups

Recently we've had a bit of a crisis situation when one of our Azure VM's decided to lose a bunch of data. Fortunately backup jobs were set up through Azure's Recovery Services and I've already used that a few times to restore or make copies of various VM's without any problems. A few clicks and you're ready to go. This was supposed to be an easy 20 minute task but this time was different.

For whatever reason instead of getting a restored VM, I started getting the following restore job fail message:

Restore failed with an internal error.
Please retry the operation in a few minutes. If the problem persists, contact Microsoft Support.

Not very descriptive, and not really helpful. :/

The data transfer part of the job would succeed each time, but the "Create the Restored VM" kept on failing. I tried using different restore points from a day, week, or even a month back, but it made no difference. It all came to a point where we had to submit a ticket to Microsoft to resolve this issue.

The two possible solutions that were presented to us were:

  • either restore the VM under a new Azure Cloud Service - this worked fine, but wasn't really what we wanted to do (you don't really want to pile up additional Cloud Services just to do a simple restore, it makes no sense and leaves a messy infrastructure behind)
  • restore the VM through Azure Powershell - this was a bit trickier, but it worked great in the end

So after a bit of research I realized that the Azure Web Portal doesn't actually use the exact same back-end infrastructure as Powershell which is a bit weird and should probably be emphasized a bit more throughout Azure documentation.

Microsoft support told us to follow this documentation page to restore the VM using powershell, but the tutorial wasn't without its kinks either.

Perhaps this got resolved by now but for the whole thing to work, I first needed Azure Powershell v1. That ended up being a bit of a pain because it required the regular Powershell v3 where Windows 8.1 comes with Powershell v4 and the downgrade was another mission impossible... In the end I somehow managed to resolve this issue by installing the latest Azure Powershell using Microsoft Web Platform Installer. That gave me the much needed Azure tooling for Powershell. Yay!

Now to code - these few Powershell commands will extract the VHD from the backup:

> Select-AzureRmSubscription -SubscriptionName YourSubscription
> $backupvault = Get-AzureRmBackupVault -Name "YourBackupVault"
> $backupitem = Get-AzureRMBackupContainer -Vault $backupvault -Type AzureVM -name "YourVmName" | Get-AzureRMBackupItem
> $rp = Get-AzureRMBackupRecoveryPoint -Item $backupitem
# change the $rp number to select the recovery point you want here
> $restorejob = Restore-AzureRMBackupItem -StorageAccountName "yourStorageAccountName" -RecoveryPoint $rp[6]
> $restorejob = Get-AzureRMBackupJob -Job $restorejob
> $details = Get-AzureRMBackupJobDetails -Job $restorejob

From here I finished the process using the Azure Portal as the rest of the process / powershell commands from the documentation seemed to be out of date and didn't work.

To complete the process, you should go to the Azure Portal, to VM section, and then select the "Disks" tab. From there you'll be able to create an unprovisioned disk which you will then use to create a new VM from. Afterwards click the + icon in the bottom left corner, and choose "create a VM - from gallery", you will see an option to use your newly created disk. Finish the setup and you're good to go. :)

Hope this helps you if you find yourself in a similar situation. Cheers!


Install sassy studio and ruby to compile your sass files quicker

Until yesterday I was using a free version of Mindscape Web Workbench to handle my SASS files and compile them into CSS. Over time, however, as the CSS in projects became more complicated and the files became larger, Mindscape just wasn't quick enough. Upon the guidance of Kit, I decided to switch tools and move to Sassy studio.

Sassy Studio relies on Ruby (Sassy uses a Ruby library to compile the SASS), so you'll need to install that too. The order in which you install these tools does not matter. You might already have Ruby available on your machine depending on your skillset. My current Visual studio version is 2012, you might need to check your versions to find a compatible extension.

Install sassy studio

You can install it from https://visualstudiogallery.msdn.microsoft.com/85fa99a6-e4c6-4a1c-9f00-e6a8129b6f4d.

Install ruby

You can install that from http://rubyinstaller.org/downloads/.

Configure

Once you have both installed, boot up Visual studio and check your settings (The ruby path is important, the rest is up to you). Here are mine:

Sassy Studio Settings

That's it, done. My SASS is compliling much quicker, and I don't need to wait as long to refresh the page when I'm "Making the logo bigger". Kit tells me about faster tool using C, which uses time travel to compile your CSS, but for now I'm happy with Sassy Studio.


Should you use Angular2 in production?

Recently we had a small debate about Angular2 and what the benefits and pitfalls of using it for a project right now would be. In the end Fabrizio and I came up with a short list of pros and cons.

Pros

  • Typescript will force developers to write better code.
  • Angular2 should be faster than the Angular1.
  • It is best not to invest in a framework if it is to be shortly discontinued.
  • You will be one of the Angular2 pioneers.
  • The development process will be very strict and it will require a good knowledge of the project.
  • Localization of application will be easier with the implementation of the shadowDom.
  • Debugging templates will be easier because they will raise runtime exceptions.
  • The code needs to be built before deployment. This will slow down the process but will spot code errors and typos.
  • Gaps between browsers implementations of new standards will be handled by specific libraries (Angular2 will emulate the shadowDom).

Cons

  • It is in an Alpha version. It means that the inner structure could (and it will) be subject of big breaking changes.
  • The API is not stable yet (breaking changes will be introduced).
  • Not all features are implemented yet (you will have to reinvent the missing parts and then once they get officially implemented, your custom workarounds will be obsolete and probably not as optimized and not as good as Angular2).
  • Not enough documentation. Also not enough code examples on the web, so much work will be pioneering.
  • Ecosystem is not there yet (not all libraries and tools are ported yet). For example: there is an alpha of Bootstrap; Foundation isn’t there yet; the router is not ready yet. The lack of availability of convenient libraries may mean more development.
  • Both versions will remain on the market and both of them will be actively developed.
  • The team is still thinking about "how to do things for Angular2".

So that's what we came up with. Of course there is no ultimate answer and surely Angular2 will be a good tool once it's ready. But before that happens, we think it's probably best not to use it for serious projects that need to go into production.

Speaking of framework readiness here is an appropriate comic from Commitstrip that hits the spot.

undefined


Flexing our CSS muscles

undefined

I just recently started working on a small pet-project to keep sharpening my skills on the whole stack in my free time. One of the things I decided to go with is angular-material. It's an official Angular implementation of Google material design and it uses flexbox layout instead of the grid layout we're used to seeing in frameworks such as Bootstrap or Foundation. We talked a bit over it and since we will most probably use angular on most of our future projects, we decided to keep an eye on the whole thing so we could start using it when it feels production-ready.