Nasty Bite From the Apple

Apple have gone bonkers. They have changed their developers license to make sure that Flash can’t run on iphone, itouch and ipad devices. Do they really have it in for Adobe Flash or are they just trying to hide the possibility that they’re going to launch their own software in competition to Flash, or maybe their devices will run really slowly if they were to support Flash?

Considering that Flash is on 98% of all desktop computers and most of the best or most visited websites use Flash this is very strange behaviour exhibited by Apple.

But you know what, I don’t care! Why? Because I don’t own any Apple kit. I never have done and I’m not sure if I ever will. You see, Apple produce fashionable portable devices and I’m not one to follow fashion, especially if it looks far better than it actually performs as far as features are concerned. (I admit the UI is good but the rest of the world is catching up – seen Windows 7 Mobile?) And denying i(pod|pad|touch) users the ability to tap into Flash is denying them a fair bit of functionality. But do the users really care? Probably not since millions of people have already spoken with their wallets.

Windows 2008 Hyper-V or Volume Shadow Copy Bug Part II

Following on from the problem I blogged about a couple of weeks ago, this is definitely a bug in Windows. From what we’ve been told it’s an inheritary, built-in limitation with VSS that prevents it from cleaning up after itself once 9,999 snapshots have been created. Therefore it only manifests if your server has created that many snapshots, so the explanation goes.

Now, not many people run snapshots frequently enough to encounter the problem but since our servers are running a backup every 30 minutes and there are 14 VHDs connected to Hyper-V we would run into the problem in just 15 days. That’s the explanation we were given but that doesn’t explain why after cleaning up the registry using devnodeclean and chkreg the registry will immediately start to bloat during the next backup. Hmmm. And we encountered symptoms (freezing) only after the first week – that’s less than 9999 snapshots. Hmm.

I’m hoping the engineers investigating this can come up with a permanent fix very soon.

2008 Server Freeze, Hyper-V or Volume Shadow Copy Bug?

We have been scratching our heads over a very strange problem for the last 4 weeks which causes two new servers to lock up for up to 2 hours after logging on after a reboot. They’re running Windows 2008 R2 with Hyper-V and Windows Server Backup roles installed.

After trying plenty of ideas to eliminate the problem it was pointed out to us by a Microsoft support guy that our System hive file was 343MB in size. It’s only supposed to be 15 to 20MB. I exported it as an ASCII file from regedit and opened it in Notepad. I counted 24,000 entries for VSS Snapshot devices! When Windows boots it tries to process 24,000 devices which causes it to choke killing the server for two hours – although the VMs limp on underneath and the host responds to pings but both the remote and local console is completely frozen.

Example registry entry:

[HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Control\Class\{533C5B84-EC70-11D2-9505-00C04F79DE2F}\0349]
"InfPath"="volsnap.inf"
"InfSection"="volume_snapshot_install"
"InfSectionExt"=".NTAMD64"
"ProviderName"="Microsoft"
"DriverDateData"=hex:00,80,8c,a3,c5,94,c6,01
"DriverDate"="6-21-2006"
"DriverVersion"="6.1.7600.16385"
"MatchingDeviceId"="storage\\volumesnapshot"
"DriverDesc"="Generic volume shadow copy"

Trying to delete the snapshots using vssadmin from the command prompt threw this error: “Error: Snapshots were found, but they were outside of your allowed context.  Try removing them with the backup application which created them.”

So the question is what is causing 1000’s of VSS (volume shadow copy) snapshots to be created? A clue was found in the system event log when Windows Server Backups runs: “Failed to delete the shadow copy (VSS snapshot) set with id '1A1938A0-1590-4BF4-8173-20DF5FD69E36' in the running virtual machine 'MGT01': Unspecified error (0x80004005). (Virtual machine ID A3F941F1-ED7F-48E9-9CD7-CB7C28A6604A)”

We’re using Windows Server Backup (WSB) to take incremental backups every 30 minutes for a bare metal restore of the host and its Virtual Machines. That’s 48 backups a day of 14 VHDs for 42 days that the servers have been running for. Do the maths and that comes to 28,000 VSS snapshots. Taking into account that some backups failed to run and we stopped backups for a few hours here and there, this tallies with the 24,000 devices I counted in the registry. Bingo!

So the bottom line is that the VSS writer creates a snapshot for each VHD at backup time but for some reason isn’t deleting the entries from the registry, although it is deleting the actual snapshots otherwise we’d have run out of disk space by now. Everything points to a bug in either the VSS writer or perhaps WSB or Hyper-V. They’re so tightly integrated during the backup process it’s hard to say which of the 3 is the culprit.

Since this problem is reoccurring on two new servers from Dell we are sure this isn’t a one-off freak incident. There is only 1 other similar incident reported on the web and that was a year ago on a HP server using BackupExec with the Hyper-V aware option. I’m waiting for Microsoft to get back to me, although I’ve been warned that even if they admit it’s a bug it could take a long time to produce a fix. We’d love to know why 1000’s of people who use Hyper-V and take frequent backups aren’t experiencing the same problem. There is no other software installed on the host apart from standard Dell drivers. Weird!

CF Admin SQL injection flaw

Nathan Mische has blogged about a surprising ‘flaw’ in the ColdFusion Administrator that’s supposed to allow you to disable SQL commands coming from CF such as delete, drop, alter, update, etc. Apparently this is ineffective against multi-line SQL injection attacks – the most common sort of injection.

Perhaps ‘flaw’ is the wrong word, but these settings do lead you to believe that any sql with delete, drop, etc in it would be prevented from being sent to the database.

The bottom line is to always use cfqueryparam or stored procedures and sanitise user input from forms and urls. Don’t trust users – assume they’re all up to no good! ;-)

ColdFusion 9 is out. Upgrade? Not this time.

After nearly a year of alpha and beta testing the latest release has hit the streets. It’s available to download as a 30 day trial or as a free developer edition (limited to 2 IP addresses as per usual). The new features are listed on the Adobe site.

The license has changed for the better which allows you to run the full version of CF on your development and test environments for free providing you’ve bought a new CF9 license for your production environment. Therefore if you were previously limiting your dev and test systems to 2 pesky IPs you can lift that restriction, but the new license only applies to CF9, it cannot be retrospectively applied if you own CF8 and earlier editions. Well done Adobe!

The question everyone is asking is shall I upgrade our existing servers to CF9? I made the decision a while ago to skip this edition. You have to ask yourself what business benefits it will bring to the table and if that will enhance your apps sufficiently to pay for itself. Here’s my take on the new features. Obviously it’s different for every app and every business, but needless to say our new apps being launched in the months to come will be using CF9, but our existing ones will not.

• cfspreadsheet: Let’s you read from Excel files and update them too. Good if you work with xls files. I can see benefits for intranet apps. Previously to create xls files you had to create data as an HTML table but CF8 was unable to read or update existing Excel created files.
  
• Word to PDF: I really like this conversion facility but it does not officially support Office 2007 or 2010 docx files which is somewhat of an oversight. However, it will make a reasonable attempt at converting them but chokes on the more elaborate docx files as explained in this blog. OpenOffice needs to be installed for this feature to work. (While OpenOffice is free I’m not comfortable installing a bloaty desktop app onto production servers)
  
• ColdFusion as a service: Access CF features such as cfhart, cfdocument, cfimages, etc, as a web service. Perfect for offloading tasks to other servers or opening up CF features to .NET or PHP apps.
  
• Adobe AIR database synchronisation: If you’re into AIR this sounds like a useful addition.
  
• Virtual file system: Save files to RAM as if it were an ordinary hard drive. Good if you have a load of files that need to be regularly written to or read from or if your network drives are already a bottleneck for performance. Don’t forget all files in the virtual file system are wiped when the server is rebooted or crashes.
  
• Integration with CF Builder: If you’ve tried the beta of Builder (the new IDE from Adobe) then it will work better if you have CF9. I don’t like Builder. Sorry. I wish they had enhanced Dreamweaver instead because I also need access to design tools as well as coding tools, plus DW has a very good GUI.
  
• ORM: It sounds clever – CF will interface with your database without having to write a single line of SQL. Admittedly I haven’t got into this too deeply but I fail to see how it can write SQL as intelligently as a human or work with the many complexities of our stored procedures and data intricacies. I like to know exactly what’s happening at the CF<>SQL layer so ORM would worry me. Maybe ORM is for non-enterprise apps and RAD? I’m sure someone will enlighten me.
  
• New AJAX controls: I have a real problem  with this. Why are Adobe wasting their time adding a CF layer for JavaScript when the JavaScript library becomes outdated within a matter of months? If developers use a highly flexible JavaScript framework such as jQuery (which is really easy to learn and oh so sexy) then they can always keep their apps up to date with the latest features instead of having to wait a year or two for the next CF update (which also updates the built-in Ext JS library). jQuery is less bloaty and give developers much finer control over ajaxy things, ready-to-go menus, slick UI panels and interactive data tables etc.
  
• SharePoint integration: This is fantastic if your business or client uses SharePoint. It makes it so much easier for CF apps to take part or become the hub of new SharePoint apps. My biggest client has made a massive move to SharePoint which would have got me excited if they hadn’t banished all non-Microsoft technology from their organisation. Doh.
  
• Server Manager: Administrate multiple CF servers from a central console. Very useful if you have a large server farm or regularly tinker with the Administrator on a couple of servers. Definitely good for rolling out new CF servers. Otherwise check out Merlin, an AIR based administrator for managing multiple servers running CF 7, 8 or 9.
  
• Enhanced Flash Remoting: Never used Flash Remoting or Flex, I am not a Flash developer as I find using DHTML/jQuery fast and effective for enhancing the user experience, but obviously this is welcomed by those who use Flash Remoting. (I wonder what percentage of CF developers do use it?)
  
• Speed enhancements: Other blogs have run tests to show that CF9 is faster than CF8. So if your current server is under strain maybe you can buy it some breathing room by upgrading? But realistically, new server hardware may be cheaper than a CF upgrade and if your server is 3+ years old a new Intel 55xx based server could quadruple your server’s speed and/or capacity. (We’re going down the hardware upgrade route)
  
• 64 bit Edition for CF Standard: Yay!
  
• Cache enhancements: The popular Ehcache technology is now integrated into CF. Rob Brooks-Bilson talks about this in detail over 4 blog entries so I’ll leave him to explain since he’s done such a fantastic job. Part 1. Part 2. Part 3. Part 4.

Finally, Adobe have produced a useful product matrix showing the differences between CF7, 8 and 9 for both Standard and Enterprise editions of each version.

Validity – flexible jQuery form validation

I was looking for a form validation plugin for jQuery that offers enough flexibility to hook in to my own rather complex forms and existing JavaScript. After auditioning several plugins I came across Validity which has seriously impressed me.

Validity, by Wyatt Allen, offers the key following features:

1. Built-in validators for email, number, url, date, range, length, etc.
2. Equal(), distinct(), sum(), etc, methods for advance validation.
3. Use jQuery selectors to create rules for which fields to validate and compare.
4. Easily extendable using your own regular expressions or js functions.
5. Fully chainable validation. e.g. $("#lastname").require().minLength(2).maxLength(40).nonHtml();
6. Optionally provide your own validation error messages.
7. Multiple validation error display modes: Creates neat arrowed validation messages by each field with an error, or Modal mode, or Summary mode, or create your own! e.g. javascript text based alert, or outline error fields in red, or display errors at top of page, etc.
8. Automatically picks up the name of each form field or use a friendly name with the title attribute. e.g. <input type=”text” id=”firstname” title=”First Name”>
9. Validity can intercept the submit button or be called when you want it to (if you use ajax for example).
10. Only 9KB in size

Some example uses:

$("#title").require().minLength(25).maxLength(250);
The title field is required, min char length is 25 and max length is 250.

$("#duration").require().match("integer").range(1,400);
The duration field is required, must be an integer, and numbers from 1 to 400.

$("#email,#email_confirm").require().match('email').maxLength(70).equal("Email addresses do not match");
Applies to both the email and email confirmation fields, both required, must be a valid email address up to 70 chars, and must be equal to each other. My own error message will be shown if they are not equal.

$("#postcode").require().match(/^([a-z][a-z]?\d\d? ?\d[a-z][a-z])$/i,"Postcode is invalid");
The postcode field is required and must match my own regex for UK postcode validation and will display my own message if it’s invalid.

Here’s the documentation with a couple of little demos. The home page is here and you can download the plugin from Google Code.

Well done Wyatt, this is a brilliantly thought out plugin leaving ample room for developers to customise their own validators, messages, and display rendering. Perfect! :-)

Windows 7 Annoyances & Backups

Up until now Windows 7 was just running on my laptop, a non-critical machine I use to run presentations on. Now that the final code is available to developers and my shiny new Intel SSD “G2” drive has arrived I thought it was a perfect opportunity to upgrade my main desktop PC to Windows 7.

It was a fresh install using the new SSD drive. It took 4 attempts before I had a stable install; the installer hung once, after one successful install the drive refused to boot up, the “repair Windows” option failed, yadda yadaa. 24 hours after it all appears to be working my fingers are still crossed.

Now, onto the issue of this post. When I shut down my PC at night I want to carry on working where I left off the following day – at least I want all the folders to be open up at the same place. Therefore, on XP I had both these options checked in Tools/Folder Options: “Restore previous folder windows at login” and “remember each folder’s view settings”. Windows XP obeyed this command like a faithful dog, but Windows 7 doesn’t play ball. On rebooting it opens all my previous windows on top of each other, in a pile. What a mess! I then spend a minute rearranging them all, dragging them to their correct position and resizing them to how they should be. What a pain!

Furthermore, Windows 7 forgets they layout options for each window. The option “remember each folder’s view settings” that was in XP is missing in 7! A couple of my windows list files that I frequently access so I don’t want to see the navigation pane, just the file list. I use another window for managing files so I always want the navigation pane displayed. XP obliges by 7 is so stupid it can only remember 1 folder setting and applies it to all of them, so on reboot the windows are either all with navigation panes or without, not the customised mixture that XP used to respect.

For those reasons I find Windows 7 very frustrating. Someone suggested using hibernate or suspend instead of shutting down. That may be okay if Windows doesn’t refuse to recover from hibernation, but it shows an error on resume and reboots.

Remembering folder views and locations is a very, very simple function for an OS but Windows 7 has taken a huge leap backwards here. It does not compute. Windows 7 has been given the memory of a goldfish.

Backups – Aargh!

I want to back up my C (boot) drive which is 80GB onto an external 200GB drive. No, it can’t do that because Windows has decided that the backup has to include my D drive which is a massive 1TB internal drive. What? I can’t back up my boot drive because Windows insists that the D drive is also included? How stupid is that. I do not want to back up my entire 1TB “D” drive, just the critical C drive.

Admittedly I have installed non-essential apps onto the D drive such as Firefox, Visio and Dreamweaver, plus I changed the location of My Documents from the C drive to the D drives to save disk space on the precious 80GB SSD. But those files aren’t critical to restoring Windows should the need arrise.

I’m now testing out other backup options. My key requirements are:

• Volume Shadow Copy – so open files can be backed up like Outlook’s .pst files
• Back up specific files from any of my drives without having to backup the entire drive
• Option to back up files into zip files so the backups are completely independent and not proprietary files formats.
• Additionally create an image backup of a drive (to quickly restore my boot drive onto another drive should it die)
• Incremental backups so only changes are backed up on a daily basis
• Windows 7 bit-bit compatible

Cobian Backup has always been perfect for file backups (it’s a brilliant free app) but sadly it’s only a 32 bit app which means it can’t do volume shadow copies on a 64 bit OS – which is what I’m now running to utilise the full 4 GB of ram in my PC.

So I will now evaluate the following free or cheap alternatives:

1. Acronis True Image Home 2009 (commercial, £40/$60, or half price here)
2. GFI Backup Home Edition (free)
3. Macrium Reflect Free Edition (free)
4. Macrium Reflect (commercial, £20/$30)

Acronis is not yet fully Windows 7 compatible which is a shame as it has some very powerful features. GFI Backup sounds great as a file-only backup solution (it doesn’t do drive imaging, same as Cobian Backup). Macrium Reflect (commercial) does both file and disk imaging but backs up to its own proprietary file format.

I think I may use two apps for a separate disk imaging and file backup strategy.