CFScript v2 – go vote for it!

CFScript is an inconsistent mess. Some statements take the form of functions while tags like cfloop are still named as cfloop in cfscript rather than just loop. Take a look at this:

cfloop(from=1, to=10, index=1){
     //stuff
}

Huh? Why shouldn’t a more JavaScript-like syntax be used rather than trying to replicate the tag’s BASIC-like syntax as literally as possible. It’s like trying to fit a square shape through a triangular hole. Sure, if you bang it enough it might go through, but it’s messy and you know it’s wrong.

No new developer should have to learn this conflicting, inconsistent, scripting language. It should be trashed but retain 100% backward compatibility.

How? Adam Cameron raised some inconsistencies of cfscript in his blog and without much thinking I suggested something simple to allow a new, clean scripting language to be used without having to throw away legacy code:

<cfscript version=2>
      // new code goes here
</cfscript>

When the CFML engine sees that version 2 is stated a new interpreter kicks in to compile it to Java. Obviously if version 1 is stated, or nothing at all, the legacy interpreter is used.

This feature suggestion has been added to Adobe’s bugbase. Please go and vote for it!

Adam picked up on my moment of clarity and blogged again with some great feedback from the CFML community.

V2 implemented this way will guarantee backward compatibility and wipe the slate clean to attract newcomers to CF and utilise it based on what they know from a more elegant, universal syntax such as JavaScript. Tagged based scripting and CFScript v1 are both a non-starter for newcomers who want to start a programming career. It's time Adobe think of the future, stop adding poorly implemented features that no one wants to use (cfclient?) and go back to the drawing board and get the language right without having to throw the baby out with the bath water.

Go vote now. You’ll need an Adobe ID to log in before you can vote.

Adobe CC – that really means Continuous Cost

Adobe have moved all their big products over to a new licence called Creative Cloud that means customers will no longer own their own copy of the software. You have to rent it now, therefore the “CC” brand has a special meaning to Adobe’s senior management – Continuous Cost. Photoshop, Dreamweaver, InDesign, Illustrator and many other favourites of creative professionals and hobbyists must be rented from now on. Worse, ColdFusion support for Dreamweaver has been ejected.

So what’s the big deal and the bottom line with the new licencing model?

• Pay a monthly fee to rent a product
• If you stop paying the software will stop working (that’s why Adobe connects their software to the cloud)
• It’s much more expensive for people who don’t normally buy an upgrade each year
• If you don’t like Adobe’s new way of doing business then tough, it’s Cloud or nothing from now on
• If you’re a small business and fall on hard times you must choose between buying food and electricity or your monthly Adobe licencing bill. If you choose the former then your Adobe software stops working and you can’t do any work at all.
• For students, you too must find money to pay Adobe every month.

Example Pricing

Single product monthly cost = £17.58 x 12 months = £210.96 per year.

Upgrade from CS3+ to latest Cloud version = £8.78 x 12 months = £105.36 for first year only. For the second year onwards the price reverts to the full cost of £210/year or whatever it is at the time.

If you’re one of those people who religiously bought every upgrade for the desktop product then you might not experience a price increase moving to the new licencing model (you get every upgrade when you rent from the cloud). But if you only bought an upgrade when a must-have features has been added, say every other year, then you will be paying nearly double what you previously paid just to use the product on a day-to-day basis.

As usual, UK and European customers pay a premium over US customers. The respective monthly prices are $19.99 (£13.42) and $9.99 (£6.70). So an upgrade for the first year costs £25 more for the first year and £50 more for each subsequent year. That’s a 25% premium for not living in the USA. How is that justified if all the software is provided from the cloud. Do European customers drain 25% more resources from Adobe than US customers? The cloud downloads are probably hosted in the US but even if they were in Europe the hosting costs wouldn’t be different enough to warrant charging £50 extra for a single annual download!

If you need to use at least 3 Adobe products then you can get better value from their full suite. For £46.88 per month (£562/year) you’ll have access to every Adobe product they make. It’s unlikely you’ll use even half the 24 products that are included, and even if you did you’ll have to keep paying £562/year forever if you want to access your Photoshop, Illustrator, or Premier files any time in the future.

I used Premier a lot 10 years ago and if I needed to access the project files I can load them up using my old copy that I still own. But if the Cloud licence had started 10 years ago I would suddenly have to start paying a lot of money again just to load up one of my old files. Think about it. Anything you create with Adobe CC and save in one of Adobe’s native file formats can’t be accessed again in the future if you’re not paying monthly to rent the software back. If you want it just for one month only so you can look at or convert your files, you’ll need to pay a greater premium of nearly double the price!

Adobe Not Listening

Adobe’s Photoshop blog has a post called “Creative Cloud – we’re listening” with over 300 comments that Adobe have not listened to. And we say American’s can’t do irony? One comment tests whether Adobe are listening by writing “Wait, they don’t even moderate these comments? F**K, S**T, B*LLS?” There are no asterisks in the original. I wonder how many more weeks it will remain there, unread and unmoderated by Adobe.

Alternative Software

So what can we creative people do about it? Complain to Adobe? Ha-ha, don’t waste your breath. Adobe did a Van Gough to themselves years ago. Start to look at alternative software. Adobe aren’t the only players in the market. I admit most of their software is excellent and people will roll their eyes at the thought of having to learn to become proficient with alternative software. But there’s a lot of other good products out there that will save you money and send a clear message to Adobe’s greedy owners.

Lifehacker has an article suggesting which software you can download for free or for less for each of Adobe’s main products.

Mac owners should check out Gigaom’s article on cheaper alternatives.

A useful site is alternativeto.net that lists potential alternatives to all sorts of software such as Photoshop, Illustrator, Indesign, Dreamweaver and anything else you can think of. You can filter by OS and type of licence (commercial or free). Obviously not everything listed is good, or even satisfactory if you’re used to the Adobe version, but some options might come close if given a chance.

Dreamweaver CC Degraded

If the licence issue wasn’t enough to give developers unwanted headaches, then you might be gutted to hear that support for ColdFusion and .NET has been ripped out of Dreamweaver CC. Only PHP support remains. You’d think Adobe would retain support for their own server-side language. This has infuriated creative web designers who like the visual strengths of Dreamweaver to design and maintain their sites while incorporating CFML code. The alternative Adobe product, CF Builder, is not on par with Dreamweaver when it comes to HTML design tools. There is a glimmer of hope. I heard that DW CC will let you manually add the .cfm and .cfc file types back and will still syntax highlight CFML and provide auto-complete code hints.

Adobe release details of next 2 ColdFusion releases

Since Adobe acquired ColdFusion from Macromedia in 2005 they’ve been pretty tight lipped about future releases. There’s always been speculation about the future of the product, but Adobe have just shown the first signs that they’re changing their ways and giving ColdFusion developers a really big boost.

What has triggered this change at Adobe? Maybe because their accountants noticed the last quarter was the most successful for ColdFusion since 2008? “More of that please!” I hear them shout from the top of the tower! :-)

First off ColdFusion “Splendor” (that’s its code name for version 11) will be released in 2013 followed relatively quickly by “Dazzle” a year later. Adobe have committed to faster development cycles and lots of new features:

• Streamlined mobile application development
• Revamped and new PDF functions (at last, yay!)
• Easy social media integration for Enterprise
• Improved installation and deployment
• Enterprise performance & scalability
• Responsive multi screen content (erm, that’s client-side stuff)
• Social analytics
• Enterprise video portal (whatever that is?)

Read all of Adobe’s announcements about CF at their blog and checkout the roadmap PDF too.

Well done Adobe, no one expected to hear so much about the next release, yet alone the release after that as well!

Critical CF hotfix must be applied pronto!

Adobe released a security hotfix on 10th August 2010 and classed it as “important”. However, if security is even moderately important to your ColdFusion server, the file system, database and network then you must think of this hotfix as CRITICAL. Just to clarify, this is CRITICAL. An HP security expert has blogged about it and It also caught the eye of The Register.

Hackers have proven how easy it is to use a vulnerability in CF 7, 8 and 9 to gain access to the CF Administrator. Code to perform the hack has been made freely available (which I won’t directly link to, but others have). Mike Bailey tweeted “It works and it’s scary.” Someone else chimed in and showed how you don’t even need to hack the Administrator’s password by using a cheeky bit of JavaScript.

Why is this really bad? Well, once you have access to CF Admin you can run scheduled tasks to access the OS. Someone has kindly(?) written an FAQ explaining how it works and why it’s so bad.

Now that the world knows how to hack it, everyone running CF must now patch their server. Adobe need to hammer home the seriousness of this problem and how critical the hotfix is. “Important” doesn’t stress it enough.

If you have already made the “administrator” directory inaccessible to the Internet or IP protected it then you should be safe, but it’s a good idea to still apply this critical hotfix.

ColdFusion Bugtracker Bug

How does one submit a bug about a bug submitting tool? Below, the ColdFusion Bugtracker after completing a 2 page form.

I tried this twice, logging in again the second time just to make sure I was logged in. Booo. Now I have 2 bugs to report to Adobe!

Sessions never expire bug in ColdFusion 8.01

A couple of times a year I’ve encountered a strange problem in our ColdFusion servers where sessions mount up and aren’t removed after they should have expired. Just today we had 100’s of sessions left in memory with all of the session scope variables still there at the end of the day, hours after they should have been deleted. Automated session housekeeping ceased to be.

Two other symptoms drew my attention to the above problem which must surely be related. Emails stopped being sent, the spool directory filled up without any cfmail files leaving. And from the website CF sporadically threw the error “The session is invalid” which was temporarily resolved by closing the browser and logging into the app again.

Restarting the CF services doesn’t resolve the situation because the service refuses to stop if asked politely. A full reboot is the only way to restore normality with confidence.

You’re probably thinking what good is it complaining now because 8.01 isn’t the current release and we should upgrade to 9.x. Well how do we know that 9.x has fixed this problem? Are Adobe aware of the issue? We could run a 30 day trial of 9.x on a test server but we’d have to run it for at least 6 months with a constant load to mimic our production server keeping in mind the rare appearance of the bug.

If you have encountered this problem before or know how to fix it then please let me know.

CF Admin SQL injection flaw

Nathan Mische has blogged about a surprising ‘flaw’ in the ColdFusion Administrator that’s supposed to allow you to disable SQL commands coming from CF such as delete, drop, alter, update, etc. Apparently this is ineffective against multi-line SQL injection attacks – the most common sort of injection.

Perhaps ‘flaw’ is the wrong word, but these settings do lead you to believe that any sql with delete, drop, etc in it would be prevented from being sent to the database.

The bottom line is to always use cfqueryparam or stored procedures and sanitise user input from forms and urls. Don’t trust users – assume they’re all up to no good! ;-)

ColdFusion 9 is out. Upgrade? Not this time.

After nearly a year of alpha and beta testing the latest release has hit the streets. It’s available to download as a 30 day trial or as a free developer edition (limited to 2 IP addresses as per usual). The new features are listed on the Adobe site.

The license has changed for the better which allows you to run the full version of CF on your development and test environments for free providing you’ve bought a new CF9 license for your production environment. Therefore if you were previously limiting your dev and test systems to 2 pesky IPs you can lift that restriction, but the new license only applies to CF9, it cannot be retrospectively applied if you own CF8 and earlier editions. Well done Adobe!

The question everyone is asking is shall I upgrade our existing servers to CF9? I made the decision a while ago to skip this edition. You have to ask yourself what business benefits it will bring to the table and if that will enhance your apps sufficiently to pay for itself. Here’s my take on the new features. Obviously it’s different for every app and every business, but needless to say our new apps being launched in the months to come will be using CF9, but our existing ones will not.

• cfspreadsheet: Let’s you read from Excel files and update them too. Good if you work with xls files. I can see benefits for intranet apps. Previously to create xls files you had to create data as an HTML table but CF8 was unable to read or update existing Excel created files.
  
• Word to PDF: I really like this conversion facility but it does not officially support Office 2007 or 2010 docx files which is somewhat of an oversight. However, it will make a reasonable attempt at converting them but chokes on the more elaborate docx files as explained in this blog. OpenOffice needs to be installed for this feature to work. (While OpenOffice is free I’m not comfortable installing a bloaty desktop app onto production servers)
  
• ColdFusion as a service: Access CF features such as cfhart, cfdocument, cfimages, etc, as a web service. Perfect for offloading tasks to other servers or opening up CF features to .NET or PHP apps.
  
• Adobe AIR database synchronisation: If you’re into AIR this sounds like a useful addition.
  
• Virtual file system: Save files to RAM as if it were an ordinary hard drive. Good if you have a load of files that need to be regularly written to or read from or if your network drives are already a bottleneck for performance. Don’t forget all files in the virtual file system are wiped when the server is rebooted or crashes.
  
• Integration with CF Builder: If you’ve tried the beta of Builder (the new IDE from Adobe) then it will work better if you have CF9. I don’t like Builder. Sorry. I wish they had enhanced Dreamweaver instead because I also need access to design tools as well as coding tools, plus DW has a very good GUI.
  
• ORM: It sounds clever – CF will interface with your database without having to write a single line of SQL. Admittedly I haven’t got into this too deeply but I fail to see how it can write SQL as intelligently as a human or work with the many complexities of our stored procedures and data intricacies. I like to know exactly what’s happening at the CF<>SQL layer so ORM would worry me. Maybe ORM is for non-enterprise apps and RAD? I’m sure someone will enlighten me.
  
• New AJAX controls: I have a real problem  with this. Why are Adobe wasting their time adding a CF layer for JavaScript when the JavaScript library becomes outdated within a matter of months? If developers use a highly flexible JavaScript framework such as jQuery (which is really easy to learn and oh so sexy) then they can always keep their apps up to date with the latest features instead of having to wait a year or two for the next CF update (which also updates the built-in Ext JS library). jQuery is less bloaty and give developers much finer control over ajaxy things, ready-to-go menus, slick UI panels and interactive data tables etc.
  
• SharePoint integration: This is fantastic if your business or client uses SharePoint. It makes it so much easier for CF apps to take part or become the hub of new SharePoint apps. My biggest client has made a massive move to SharePoint which would have got me excited if they hadn’t banished all non-Microsoft technology from their organisation. Doh.
  
• Server Manager: Administrate multiple CF servers from a central console. Very useful if you have a large server farm or regularly tinker with the Administrator on a couple of servers. Definitely good for rolling out new CF servers. Otherwise check out Merlin, an AIR based administrator for managing multiple servers running CF 7, 8 or 9.
  
• Enhanced Flash Remoting: Never used Flash Remoting or Flex, I am not a Flash developer as I find using DHTML/jQuery fast and effective for enhancing the user experience, but obviously this is welcomed by those who use Flash Remoting. (I wonder what percentage of CF developers do use it?)
  
• Speed enhancements: Other blogs have run tests to show that CF9 is faster than CF8. So if your current server is under strain maybe you can buy it some breathing room by upgrading? But realistically, new server hardware may be cheaper than a CF upgrade and if your server is 3+ years old a new Intel 55xx based server could quadruple your server’s speed and/or capacity. (We’re going down the hardware upgrade route)
  
• 64 bit Edition for CF Standard: Yay!
  
• Cache enhancements: The popular Ehcache technology is now integrated into CF. Rob Brooks-Bilson talks about this in detail over 4 blog entries so I’ll leave him to explain since he’s done such a fantastic job. Part 1. Part 2. Part 3. Part 4.

Finally, Adobe have produced a useful product matrix showing the differences between CF7, 8 and 9 for both Standard and Enterprise editions of each version.

8x ColdFusion Hot Fixes in just 2 weeks!

First there were 7 hot fixes rolled out on the same day on 17th August – all to do with security vulnerabilities. Adobe’s documentation was sparse causing problems and lots of questions to be raised by early adaptors of the hot fixes.

Today, 1st Sep, Cumulative Hot Fix 3 has been released for CF 8.01 that fixes 21 new bugs as well as the fixes contained in the previous 2 cumulative hot fixes. This does not include fixes for the aforementioned 7 security vulnerabilities.

I can’t say I’ve encountered any of the bugs listed for Hot Fix 3 and they’re not security related so we’ll put some thought into whether to deploy it or not as I’m a great believer in “if it ain’t broke, don’t fix it”. Do read through the list of fixes to see if any issues are relevant to you. Maybe some of the descriptions will explain some strange errors you’ve encountered with your apps?

I notice there’s no fix for the mail spool bug I reported over a year ago.

Now, back to the 7 security hot fixes. The original documentation was, let’s face it, terrible. But on 28th Aug Adobe updated the text to make it clear that hot fix 1876 must only be applied if you’re running Apache. Do not apply it if you’re running IIS which is what I did on a test box. Luckily I held back on applying it to the prd servers before receiving confirmation that it’s not for IIS. The test box happily accepted the hot fix anyway and doesn’t seem any the worse for it.

Hot fix 1875 and 1878 are byte for byte absolutely identical which is really weird. Why didn’t Adobe roll them into the same hot fix instead of listing them separately and making people install two hot fix files which are the same in all but file name?

3 CFML Engines, a hard choice to choose

With 3 great CFML engines to choose from, the choice for developers can be a painful one. Torn between Adobe ColdFusion, Railo 3.1 and Open BlueDragon? Yeah, me too.

The core CFML tags and functions are well supported by all 3, with mostly the bleeding edge features that separate them along with the support options if you’re concerned with needing expert help if you find a bug or quirk that becomes a showstopper.

The other consideration is the platform you have to host the CFML engine. Adobe ColdFusion is well supported by hosting companies if you’re not hosting it yourself, but the others have limited options at the moment.

UPDATE: I just read that OpenBD can run on the Google Apps Engine! Live demo! Look at the appserver value. (It’s not available on there yet… work in progress…)

Most of my projects are self-hosted so I don’t care about the platform, I just need stability, easy-to-get support (free or paid-for), and a promising development roadmap.

I’m not quite finished… each CFML engine have their own unique features. Take Railo with its CFVIDEO tag for example, for someone that could be a deciding factor. Creating YouTube sites could be as easy as pie. Open BlueDragon works natively with Amazon’s SimpleDB and can pull & push files to/from Amazon S3. ColdFusion 8 has built-in MS Exchange support (which is cool) and AJAX stuff which I don’t really care for. jQuery is the way to go. :-) But ColdFusion 9 is scheduled to be released by the end of the year and will arguably leapfrog the other two with a ton of new features.

I haven’t been too happy with the support from Adobe when I discover bugs and their 2-year product cycle means some issues don’t get addressed for a long time, although their intermediate patches are warmly welcomed.

A price to pay…

Then there’s the pricing issue. Sure, £6000/$7500 isn’t a massive price for big companies wanting to run with CF Enterprise, but small companies or personally financed start-ups can’t chuck that sort of money into application software when it costs the same amount to purchase new server hardware and host it for 4 years. Even on the enterprise level if you need to expand a cluster and whack in an extra web server it’s £1000/$1500 for the hardware and then £6000/$7500 on top. A large organisation would still ask why it’s costing that much to pop in an extra server.

The current economic climate has put pressure on many businesses to cancel or scale back on projects and the awful £/$ exchange rate has made CF a lot more expensive in the UK. So maybe now is the time to look closely the open source CFML engines.

AJAXbouncer – limiting Ajax tampering and leeching

Yesterday I mentioned a proof of concept to try to stop script kiddies and data leechers from abusing server-side scripts that are intended to serve XMLHttpRequests (XHR or AJAX). Playing with URL or form parameters can get the server to return all sorts of data, sometimes even data that the developers didn’t intend you to have access to. The problem is that servers can’t tell the difference between a normal web page request and XHR.

Ray Camden blogged about how jQuery adds an extra HTTP header to help the server tell the difference, but headers are very easy to spoof.

My idea is for the server to issue the web page with an encrypted token. The token is the current date/time and must be sent back to the server for each and every XHR triggered by the current page. If the server doesn’t receive the token, or the token is invalid (i.e. it’s be tampered with) or the decrypted token reveals it’s older than, say, 5 minutes then the server returns a 404 error – page not found.

So, anyone who tries to submit data back to the server through dishonest means will find they get a 404 after 5 minutes. If they try to alter the token they get a 404 too. This will baffle script kiddies or hackers and hopefully they will move on to mess with someone else’s website. If they persist they will realise that they can’t generate their own encrypted token but will have to refresh the main web page every 5 minutes to obtain a new token and insert that into their script. That’s the only weakness in this concept, but taking it a step further you could log the IP from the first failed XHR and block serving that IP for the next 30 minutes. Or refuse to issue a new token within the same session or to the same IP.

As for genuine users you can set the web page to auto-fresh every 5 minutes. It will work best on sites where you don’t expect users to linger on the same page for too long, but of course you may prefer a longer token life (like 15 minutes).

Here’s a live demo – many thanks to Ray Camden for hosting it. The demo’s token will expire after just 90 seconds. The POST data is exposed in a grey area at the bottom of the page so you can tamper with the parameters to see what happens. The demo uses jQuery for XHR, of course.

I’ve commented the code so developers using PHP, .NET, RoR, etc can easily adapt the ColdFusion code. Download the demo code here.

If you improve upon it please let me know.

Oh, in case you’re wondering why I called it AJAXbouncer it’s because it offers a deterrent to potential trouble makers but doesn’t provide 100% safety – like bouncers standing outside pubs and clubs.

CFMAIL spool bug in ColdFusion 8.01

After applying a patch to fix a cfmail problem in 8.01 I noticed a new problem. When creating more than 3000 or so emails in one go about 100 ended up in the Undelivr directory. There was nothing wrong with those emails, dropping them into the spool directory by hand sent them quickly on their way.

The exception.log was showing “IOException while sending message”. I reported this to an Adobe engineer in July 2008. In October the engineer found that the issue can be “fixed” by increasing the max session size in the Microsoft SMTP service. Microsoft don’t recommend increasing it too much but the more I increased it the less emails were sent to the Undelivr directory.

The bug is with ColdFusion and it appears to not behave correctly if the SMTP server closes a session because it has reached the maximum size. ColdFusion should react by simply opening up a new session and continue to send emails from where it last left off. But it doesn’t do that. This bug has been affecting a major application for nearly a year and Adobe haven’t been able to fix it.

MS SMTP logs “552 4.3.1 Session size exceeds fixed maximum session size” which is fine, it expects the app at the other end (CF) to continue by opening a new session. I’ve now increased the session limit to a very large amount but as the application is used more the limit is reached and emails stop being sent again. CF is misbehaving and is causing operational headaches.

Has anyone else had similar problems? I’m looking for others who can help add some pressure to get this bug fixed. Please do get in touch. Needless to say I’m very disappointed that CF is unable to send a large amount of (legitimate, non-marketing) emails problems.

AJAX tampering and leeching

Ray Camden blogged about server side security when using AJAX. He discussed a way to detect the difference between normal HTTP requests for normal web pages and AJAX triggered HTTP requests. To the server they look virtually the same.

The problem with using AJAX is that is adds a vulnerability to web applications. People can play around with URL and form parameters to see what else they can extract from your application or database. It can also be used to leech data from your database on a regular basis because, if your data is of any value to someone else, you’re providing it freely in a machine readable format, typically JSON or XML. How can you do something to stop people do that? I mentioned a possible technique when commenting to Ray’s blog and was asked to come up with some code.

I brewed up a demo app and am seeing if I can get the live code hosted (it’s for ColdFusion but the technique applies to any server side language). Stay tuned…

If you want to play with the code on your own CF server you can download it here. No configuration is necessary, just drop the folder into your web root. The code is highly commented so PHP, RoR and .NET coders can easily modify it.