Friday, 31 October 2014

Log Parser Lizard creating custom regex format

I first heard about Log Parser Lizard (LPL) when Scot Hanselman blogged about it back in 2011 and since then its been part of my tool belt.

LPL makes analysing logs easy enabling you to point it at single log file, or even a directory of files, it will parse them and then allow you to query the data using SQL syntax.

LPL comes with a lot of predefined searches from Active Directory, IIS, event logs, etc and has the ability to allow you to define your own custom RegEx format for parsing text files.

Recently I needed to use the custom RegEx format, its not the easiest thing to do and as I couldn’t find any examples I thought I’d write up what I did to help me, and possibly anybody else that needs to do it, in the future.

Log Parser Lizard creating custom regex format

I first heard about Log Parser Lizard (LPL) when Scot Hanselman blogged about it back in 2011 and since then its been part of my tool belt.

LPL makes analysing logs easy enabling you to point it at single log file, or even a directory of files, it will parse them and then allow you to query the data using SQL syntax.

LPL comes with a lot of predefined searches from Active Directory, IIS, event logs, etc and has the ability to allow you to define your own custom RegEx format for parsing text files.

Recently I needed to use the custom RegEx format, its not the easiest thing to do and as I couldn’t find any examples I thought I’d write up what I did to help me, and possibly anybody else that needs to do it, in the future.

Wednesday, 23 July 2014

The Full Stack Developer

Back in 2010 Carlos Bueno wrote a blog post on The Full Stack Developer and since then there have been various posts that comment on it such as this post and this post which made the full stack developer seem an impossibility.

I’m not sure I buy into this idea of a Full Stack Developer as somebody who can for example throw together a web site using the latest framework/library, work at the TCP protocol level and understand i/o bottlenecks in a database system with any degree of mastery.

Whilst there are some extraordinary individuals out there that can manage this I doubt the majority of developers today can do so.

We as developers have, out of necessity, become specialists in specific areas of computing.

The rate of change, number of languages/tools/frameworks/libraries and depth of understanding required means that in general people simply don’t have the time to be able to keep up with all the changes to all the various areas of computing.

Monday, 21 July 2014

Grunt & Gulp – an overview

Furthering my journey into JavaScript up to now I have not had much interaction with JS task/build systems so to remedy that I recently attended a DoLess workshop on Grunt & Gulp to discover what they were all about.

Most often people associate both Grunt & Gulp with building systems but they are not restrict to that and both can form part of the of a front end dev’s day to day development process being able to run tests & reload html when altered, etc

Wednesday, 21 May 2014

Fashion

fashionFashions come into style and go out of style and when they’ve gone you often look at pictures of yourself and think “why on earth did I wear that”.
Its no different in software development, with various tech gaining favour and becoming “fashionable” with everybody wanting to be using that tech only to find that it drops out of fashion later.  It used to take years for things to become unfashionable but with the rate of change increasing this may drop to a few months or even weeks.

Monday, 19 May 2014

DDD South West 5

This weekend saw the return of DDD to Bristol after a break of a year, with a new location and changed team behind the organisation.

As per previous years the weather on the day was absolutely fantastic with blue skies and sunshine as far as the eye could see.

The venue

This year instead of being held at UWE the Redcliffe Sixth Form Centre which is closer to the centre of Bristol making it easier to access via Train but a little more difficult by car, but with plenty of car parks nearby I don’t think it was a big problem.

On entering the venue you proceeded upstairs into a large “common room” area which is where the attendees would be between sessions. One advantage this had over the previous venue was that the area was large enough for the sponsors to be in as well as attendees being able to get coffee/food and socialising.

My Day

On arriving at the conference I found coffee & Danishes  waiting for attendees, which was much appreciated, and whilst waiting for the welcome I took the opportunity to catch up with people I knew some of which I hadn’t see since the last DDD I attended.

Welcome session

Ross Scott gave the welcome telling everyone about the day, house keeping information, etc. One change for this year was that there would be no evaluation forms instead all feedback would be recorded by Pocket DDD and each session you entered feedback for would count as a single entry into the prize draw at the end of the day.

Session 1

I didn’t actually attend any of these sessions, none of them had piqued my interest (although having listened to people who did attend I think I may have missed an opportunity to see Steve Sandersons talk) instead taking this time to catch up with some people I specifically wanted to talk to.

Session 2 – Redis Cluster

Marc Gravell started off by explaining what Redis is, how you can install it, basics of Redis, etc.

He then showed us how simple it was to use Redis using the Stack Exchange Redis client and also touched on how to monitor Redis using the Stack Exchange OpServer which he uses daily - both of which are available for free.

Marc then covered various replication configurations for Redis with master-slave relationships before moving onto Redis Cluster.

Redis cluster is currently unavailable on windows (MS Open Tech that are porting Redis are slightly behind) Marc then explained why you’d want a cluster rather than using a master-slave configuration with a demo of how to set up a cluster which currently needs Ruby to achieve.

Session 3 – Supercharge you JavaScript Development

Chris Canal’s session was packed, people were obviously interested in the subject.

Chris had chosen to video all of the “live coding” to ensure no issues with the internet and for the most part it worked really well with Chris being able to add additional commentary on what was happening whilst the video was running.

Chris covered using npm, grunt, bower and yeomon to get a project started quickly and then covered using the lodash library in your code.

Lunch

Lunch was held in the common room area and the team had laid on pasties for the attendees which were very nice.

There were a few groks and a 20-20 presentation by Phil Winstanley, although with the common room although more people could watch it the noise from people talking and not watching did make it more difficult to hear the speaker.

By this time the heat in the presentation rooms had increased and a lot of attendees were commenting on it.

Session 4 – Hadoop and Big Data

Gary Shorts’ session was packed, again a good indicator that people wanted to know more about this subject.

Gary took us through how hadoop works and then showed us how we could use .Net to work with it using the obligatory word count demos.

Once he had explained how we could use it he managed to squeeze in a real world example, including some visualization, around trying to predict the results of horse races.

Session 5 – 10 things I learnt about web application security

James Crowley started his talk explaining he wasn’t a security expert but was passing on his experience from working on a product that had had professional penetration testers trying to break into it.

He went on to show us just how easy it was to break into a site using fiddler before moving on to more sophisticated tools such as BeEF.

James showed just how easily your site could be compromised and more than one person I talked to said they were more than a little concerned about their sites after seeing this session.

Closing Session

Ross Scott presided over the closing session, thanking the speakers and everybody for coming, and then handling the swag giveaway.

And with that it was over.

Summary

The new team put on a really good day, pasties and cream teas adding to the west country flavour of the day. I thought the new venue worked out well, especially the common room area, the only downside seemed to be the session rooms were so hot by the end of the day but it does seems this happens at every DDD SW.

I think the team can be rightly proud of what they achieved and I look forward to seeing what happens next year.

Tuesday, 22 April 2014

No Backend

In the .Net world if you were to create a web application historically you’d be developing the full stack: web pages, an API (of sorts), business logic and database.

As JavaScript MV* frameworks have become more prevalent a .Net web dev may use one of these frameworks, such as Angular or Knockout, to create a the UI but often will still be creating the rest of the stack.

As part of learning more and more about JavaScript I’ve come across the concept of “No Backend” or “backend as a service” where a company may provide all the server side functionality you might need (different companies often offering different levels/types of services).

Wednesday, 5 March 2014

4 Places to load JavaScript

I watched Enough with the JavaScript Already by Nicholas Zakas the other day and one thing that stuck with me was him outlining 4 places within your page to load JavaScript

  • In <head>
  • Before </body>
  • After page load
  • On demand

Nicholas points out that you don’t just have to load JavaScript in one place, you need to think about what your users will do on the page and this should guide where you load your script.

He then went on to offer some guidance on common things you’d load in each place:

In <head>
To determine how your users are using the site you need analytics, often those scripts state they have to be in the head e.g. Google Analytics. If you are going to put something here it should be absolutely minimal such as boot strap code. 

Before </body>
You should add scripts here that control functionality that users do immediately the page is loaded.

After page load
Once your page is loaded then load scripts that control functionality that the user may do shortly after the page is loaded scripts needed soon after page load

On demand
This is loading a script when needed specifically relating to what the user is doing. The functionality this relates to is code that is used infrequently by the majority of the users of your site (<25%).  Nicholas mentioned a blog post by Alex Sexton which talks about doing this amongst other things (also a good read).

Summary

What I specifically took away from this part of his presentation was that all too often before </body> becomes a dumping ground with scripts being put there with no real thought as to what is actually needed on the page.

Something else that crossed my mind is if you’re building a SPA you may think you’re already doing this using require.js or similar mechanism, loading only scripts as you need them, but it may pay you to take a closer look and work out exactly what is loaded and when, perhaps its not being as efficient as you think.