http://www.imovieplugins.com/B1516165238 this blog chronicles what's happening at cf/x's iMovie plug production plant. info@imovieplugins.com © cf/x Mon, 13 Aug 2007 18:49:00 +0200 Mon, 13 Aug 2007 18:49:04 Europe/Zurich iBlog 1.3.1 iMovie 08 is so different from iMovie 06 that we consider it an additional application for iLife rather than a successor to iMovie 06. this is underscored by the fact that apple has created a special iMovie 06 version that can be downloaded freely by iLife 08 owners. we look at iMovie 08 from an iMove 06 user's perspective.

on Tuesday, August 7th, 2007, Apple released a new version of iLife. bundled with iLife comes a new version of iMovie. according to Steven Jobs, Apple's CEO, the new version represents a brand new application, and 'it's not an enhancement to iMovie', but a 'completely new way of editing video'. while the latter part of his statement is debatable (we have seen our share of video editors), his former is spot on. iMovie 08 has been written from ground up as a new application, and appears to have nothing in common with it's predecessors (original iMovie to iMovie 06). it's darker, integrates with iPhoto and iTunes, supports AVCHD source, has an incredibly slick "skimming/scrubing" feature, and makes stringing together videos truly simple. on the other hand, it has no video effects, only a few transitions, abysmal audio control, and abolishes support for third-party plug-ins (such as our products). but why did Apple break with the past?

well, the reasons for this seem clear enough.
first of all, iMovie was originally written pre-OS X, and has become one of the oldest applications in the bundle. it was originally released in 1999, and came pre-installed on some Macs (most notable the iMac DV, and some high-end laptops), and ran on OS 8. now, eight years later, iMovie was getting decidedly long in the tooth, and stability issues were becoming more and more pronounced. iMovie's plug-in architecture and foundation are from a time where OS X was a mere whisper, and 'Carbon' was a radically new technology. we (cf/x) may not like that fact, but iMovie 06 is positively creaking with age, and the plug-in architecture was becoming more and more a limiting factor.

over time, Apple has added a lot to iMovie: themes, a slightly improved plug-in API, enhanced interface elements et-cetera. but all this could not obscure the fact that iMovie was inherently difficult to use, and increasingly unstable. granted, it was easier to use than any other video editor we knew - but once you tried to get serious, you ran into many obstacles, problems, and little idiosyncrasies that made working with iMovie less pleasant.

at the same time Apple made it's astonishing transformation from a computer manufacturer to the undisputed king of 'digital lifestyle'. i believe that fundamental to their success in this domain was, and still is, the relative ease of use of their devices. witness the iPod, and how it integrates with iTunes and iPhoto. no other manufacturer has managed a similar tight, seamless integration. then look at Apple TV - another nicely executed, and well integrated consumer device. or simply pick up an iPhone... during the unveiling of iLife 08, Jobs emphasized this point by remarking how the new iMac's looks and feels like a high-end consumer device. however, when bringing together music, movies and pictures in your living room, iMovie 06 sticks our as a sore thumb. it's clunky. it doesn't integrate nicely, and it's interface harkens undeniably from the last century.

therefore Apple has decided to replace iMovie 06 with a slicker, more consumer-oriented application for video editing. the result is iMovie 08. it looks more modern. it integrates better with your music, photo and video libraries. it streamlines home video production, makes sure you can get your home video done in a short time, and have it displayed on your TV or web site moments later. as such, iMovie is an enormous leap forward, a truly remarkable achievement.

on the other hand, with it's only hand full of transitions, lack of effects and downright spartan (ha! just won the bet that I can sneak a '300' reference in here) audio management, you can't get 'serious' with iMovie 08. just like many good consumer devices are well suited to cover your everyday's tasks, they often fall short when applied to a special, more demanding situation. most times, your videos (e.g. clips shot during your vacation, or of your toddler playing with the dog) can be easily edited with iMovie 08. but there will be some instances, where "08" is too limited. candidates include once-in-a-lifetime events such as weddings, graduation or retirement, and the ever popular christmas videos that people send out to their loved ones across the globe. these videos require much more care and polish but iMovie 08 does not have the necessary features for this. iMovie 06, warts and instabilities non-withstanding, does. Apple appears to have acknowledged this and has made iMovie 06 available on-line in a special iLife 08 version. without "06" there would be no application to cover the middle ground between the low-end casual iMovie 08 and low-end pro Final Cut Express. therefore, with iMovie 08 Apple has in effect added a new application to iLife. some people have called it the 'youtube editor'. while this seems somewhat derogatory, it does capture iMovie 08's essence. it enables you to quickly string together a few clips, and effortlessly share them with the rest of the world. in many respects it mirrors the iPod's astonishing simplicity. this may be what Apple TV needs to become a similar phenomenon.

some people have speculated that Apple dumbed down iMovie because it encroached upon Final Cut Express. i don't believe that for a minute. superficially, Final Cut and iMovie 06 appear to have similar features. however, Final Cut (Express and Pro) are orders of magnitude more powerful than iMovie. anyone who believes that iMovie was becoming a threat to Final Cut Express should spend a few hours working with it. Final Cut's time-neutral transitions and parallel video sources alone are features that effectively mark the difference between consumer and pro territory. there is simply no contest between these two applications.

but back to iLife 08: how will all this impact you, the veteran iMovie 06 user? here's what we at cf/x think:
*** which version should I use?
i believe that you will be using iMovie 08 on and off for quick editing jobs that do not require much finesse or your talent. just throw together a few clips, add a title, and push it to your web. every once in a while, however, you'll want to create a work of art. be it because the recipient of the video, the contents, or both are special. iMovie 08 can't help you there. you'll either fire up iMovie 06 (a short reminder to everyone that as iLife 08 customer you can download iMovie 06 for free), or you may even opt to upgrade to Final Cut.

also, users with large amounts of video will soon find out that iMovie 08 severely limits the way you can store your data. it forces you to put all clips into your iPhoto/iMovie repository (you'll have to use the correct folders in your home directory), and can't group them somewhere else. this makes sense from a 'digital hub' point of view - now Apple TV can find them and stream them anywhere. it does not, however, make sense from a producer's point of view who wishes to separate clips from one event from another, or if you keep your projects with their source material on removable storage. in this case, iMovie 08 is straight out. you must use iMovie 06 or other video editors.

*** will 08 at some point in time support old-style plug-ins?
we seriously doubt that. the old iMovie plug-in API (the programmer's interface) was clunky, based on pre-OS X technology, and not very well thought out. it pre-dates even Apple's own OS X plug-in API, and requires serious magic just to compile on our computers. it is highly unlikely that Apple will implement backward compatibility into iMovie 08.

*** will 08 support plug-ins?
we have been poking around in iMovie 08 and found out that it already has some means of plug-in support: in the application bundle we found plug-ins for various video formats. however, we have no idea how advanced this interface is, and what it can be used for. we also think that ultimately, Apple may decide against it as plug-ins add complexity. since we believe that simplicity is what iMovie 08 wants to achieve this may not be in the cards. in any event, we won't be able to produce any plug-ins unless Apple publishes the new API (interface documentation) and SDK (software development kit)

*** will Apple continue to develop iMovie 06?
we believe that Apple may produce a few stability releases for iMovie 06 should a new OS release warrant this. we seriously doubt that Apple will add any new functionality to iMovie.


so, where does that leave us (cf/x) and you (iMovie 06 users)?
ultimately, iMovie 06 will go away. this line of our business will go away, and with it our revenue stream for existing plug-ins. we'll see if and how we can develop plug-ins for 08 and beyond. this, however, requires some action on Apple's side. we will of course continue to maintain our current plug-in line. we will also, for the time being, continue porting our plug-ins for iMovie 06 to Intel. but this may end some time, and we may end the porting process before all plug-ins are ported. porting a plug-in is expensive, and with little to no expected revenue, this may not be financially viable.

what, then, will become of us? fortunately, our plug-in line of products was not our core business. better yet, during the drought of Intel transition, we have been able to develop a new application that may soon replace the plug-ins as product line. this application, however, is very different from a simple plug-in, and will take a much bigger share of our business capacity. deploying this new application will be both interesting, and a real challenge.

but until then, we'll be here, creating great plug-ins, and caring for you.

-ch

]]> Mon, 13 Aug 2007 18:17:27 +0200

spammers have always ranked low on everyone's list as people to like, perhaps an ångstrom above drug dealers. we have always despised anyone stooping so low as to send out spam, because to us they the represent filthiest, most parasitic wash-outs the internet has to live with. they try to make a fast buck out of hijacking computers, and then using these to peddle child porn, or other equally undesirable 'products'. they see nothing wrong in using other people's resources (computers, network, time) to fill your inbox with vile, unwanted trash. and from our perspective, that just about sums up their good side.

the flip side is that they also willingly create serious problems for internet-based companies. in order to fool some spam filters, spammers spoof their mail's sender using legit senders. Every once in a while, a spammer uses our 'imovieplugins.com' domain as a return adress. the result is an email that looks as if it was sent from us, but in reality was sent from a zombie PC. the only way to find this is to match the originator's email address with the IP trail of the mail, which may also contain spoofed elements. read: not simple.

so, the spammers send out a few million emails, advertising their newest penny stock scam or penis enlargement patches. the results are always the same:
1) everyone is annoyed
2) a few idiots purchase something from the spammer
3) the spammer makes a few bucks, encouraging them to do it again
4) we end up with a huge problem.

and these are the problems: first of course there are those dim-witted net admins that blindly block the spoofed originator, not the IP of the zombie PC. the effect is that after a few hours no-one who uses Comcast or Roadrunner as hosting provider will be able to receive our emails any more. it appears that these providers still choose cheap labor for their mission-critical net admin posts. so, for a large portion of the web, imovieplugins goes off the air.

the next problem is that since we can't reach customers any more their registration keys don't get through. they send us increasingly angry notes, which we can't respond to without being informed by their provider that this reply, too, was 'rejected for spam'. we now have instituted a separate (and costly) process just to deal with that problem.

another problem are the roughly hundred thousand spam bounces that are returned to us. every few million email addresses are bound to have a few percent of outdated email addresses, and all these are returned -- not to the spammer, but to us. and we have to go through every one of them manually to find the few real bounces. needless to say, this takes a lot of time.

but the worst part are the enraged emails directed at us. now, these people understandably are angry about the spam. they tell us what they think in no uncertain terms. some even threaten legal action. others might try to get us to 'change our ways'. in all cases, this is bad publicity. so we try to do what's right, and answer. we try to explain what has happened, and apologize. sometimes, this gets a positive reaction. mostly, though, we fear that our reply ends up in the spam folder, because right after the flame was sent, the sender blacklisted us. all being told, this also takes an enormous amount of time out of our small budget.

the situation has become so bad in the past months, that every time a wave of 'imovieplugin' spam hits the web, we lose about two week's worth of profit dealing with the fallout. fallout that some greedy little shit has created because they want to sell their worthless junk - and laugh about the fact that some sucker whose domain they abuse will pick up the tab.

as i write this, another wave of spam that allegedly has originated from us hits the net. the past two days saw no development on anything related to plug-ins or our other products. but some low-life probably has sold a few cheaply-made pain killers to a hapless addict.

so, if you have ever received spam from imovieplugins.com, please allow us to apologize. we did not send it. we don't send out advertisments. we don't send out product mailings. we don't even send out update notifications (as many have requested). the only way you ever hear from us if when you purchased something, or emailed us a direct question. but should you receive spam that looks as if it came from us, please don't report it as spam. just delete it. otherwise, an intellectually challenged net admin might be tempted to block our domain again.

here's to hoping that eventually, all spammers die. if possible a slow, horribly painful death caused by cheaply made medication.

because we like irony.

]]> Thu, 19 Jul 2007 21:23:49 +0200

iMovie has a lot of quirks. a major one is how it differentiates between an effect (a.k.a. 'filter'), a title (text-based effect), and a transition. rather than unifying the interface for basically three vary similar operations, iMovie's API uses three different methods to call the plug-in. the transition is the least intuitive of them all, and the SDK's examples are an exercise in bad code design.
consequently, we were loath to go through the same process we did when we developed our first (PPC-only) transition way back when. but we could not afford to put it off any longer, and started transitioning the first transition based on our reference code for the effect plug-in (at least we knew that that code worked). we ran into a lot of snags, but finally we prevailed. as of today we now not only have a reference transition running universal, but also the first transition published on our web page (pixel-pull transitions - thank you for asking). we now can port transitions and effects to Intel, and intend to bring our most important ones over as fast as possible. the new versions sport most of the same improvements as out effects, plus they now come in a single bundle (instead of three different plug-ins).

]]> Fri, 25 May 2007 17:32:50 +0200

... but the road there was rocky indeed. in the end, we had to resort to a complete re-write of the plug-in. our 'old' version relied heavily on QuickDraw and QuickTime, which do not play well with newer systems (even though simple caption was spared the fate of our chromakey line, which crashes and burns on many Macs with newer systems). still, we upgraded to Core Graphics (throwing out compatibility with older systems than 10.4 this time), and in that course were able to implement many of the things we always wanted to, but couldn't (font spacing among them).
it also presented us with a lot of new challenges, as cross-compiling (PowerPC to Intel) can become quite challenging when you tread on newer ground (for example CoreGraphics), where the API is less well tested than the classic frameworks. also, the new version drops support for non-postscript fonts, an issue that has left us scratching our heads. Quartz/CoreGraphics should be able to support these, but we are currently unable to make it work. perhaps in a later version will we restore compatibility with legacy fonts.
that was also the reason it took us so long to port a technically relatively simple plug-in - and a plug-in that, mind you, in the past few weeks was responsible for almost half of all 'urgent requests' to us for a speedy port. a short note on that: please don't request this. we port as fast as we can, but some issues take their time. resolving an arcane compiler bug in an untested environment was no fun at all, and gave rise to (perhaps justified) doubts in our sanity.
one thing we are proud of is the improved user interface for this plug-in. this was an issue that in the past has crept up regularly, and when we decided to make simple caption a complete re-write, we took the time to re-design the way text positioning and sizing works. we have integrated a lot of your feedback into this, and hope that you like it.

have a nice sunday!
your cf/x team

]]> Sat, 19 May 2007 20:01:57 +0200

for a week now we have tried to get our QuickTime movie media core to work with iMovie. the problem that is haunting chromakey and multiple movies is now in full sight. somewhere in QuickTime, there is an ugly bug, and for a whole week now we have tried to program around it. the problem is during application of the plug-in: iMovie calls the plug-in three times concurrently: once to apply to the plug-in, once to create a thumbnail for the clip, and once to create the preview. and QuickTime really, doesn't like that, crashing badly.

after coming up with a series of brilliant, but in the end ineffective work-arounds, we have for the moment conceded the ground to the bug, and turned our attention to other plug-ins that need porting. but we will get this one sorted out.

]]> Sat, 28 Apr 2007 15:27:05 +0200

the day after releasing our first universal plug-ins saw something interesting happening: we observed a sharp uptick in sales of some of our not yet ported plug-ins. while we expected a slight increase because of the publicity, this was something that at first looked counter-intuitive. a few days later understanding came in the form of an exasperated, if not unfriendly letter telling us to 'get out act together'. except not in such friendly words. what went wrong?
well, we always knew that our customers are smart (after all, they purchase from us). it seems that our pricing strategy (raising prices when we publish updated plug-ins, but offer the update itself for free) can be exploited in a cute way: purchase the plug-in you want now (at lower cost), and simply wait for the free update. this is called 'beating the system', and some people (us included) gain a feeling of smugness and joy when it works. the drawback here is of course that you don't know when a plug-in will be updated. this is called 'running the risk'. the trick here is not running the risk while simultaneously suffering from a short fuse - or worse: a deadline. because sending us emails demanding us to work harder, faster, and especially on the plug-in just purchased is not helping. so - if you think you are lucky - try beating the system, but please do not beat up the programmers.

]]> Sat, 14 Apr 2007 15:48:02 +0200

after two weeks of porting, we have managed to bring 6 plug-ins to universal, averaging at 2.3 days per plug-in. admittedly, we are currently porting the 'core family' plug-ins, i.e. those plug-ins that share a lot of code, and only require little tweaking. other plug-ins (for example the canvas line) will doubtlessly require a lot more effort to port. over the easter week-end we plan to spend one day off (searching for easter-eggs, as the custom dictates here in switzerland), and use the rest to push at least three more plug-ins out of the door, in order to complete the first ten universal plug-ins.

feedback from you has been universally positive, turning up only one slight bug. also, our price policy (charging nothing for the upgrades, but raising the prices for new purchases) has been welcomed warmly (somewhat predictably from existing customers, but we accept praise when it is offered :-)

in any event, if you have a long week-end coming up, we hope that you enjoy it, and recommend you spend some time with our new plug-ins.

]]> Tue, 03 Apr 2007 18:29:47 +0200

just how much effort was it to get the first plug-in to intel?

- getting to know xcode: 10 days
- getting to know FPC 10 days
- porting f/x core to XCode: 60 days
- adapting link layer (Pascal/C++) for dynamic libraries 20 days
- upgrading core to bundle: 10 days
- upgrading dialogs, carbon based views 15 days
- cross-compiling to i386, handling subtle differences 2 days (yes, we were surprised as well)
- upgrading interface details 3 days
- new packager, installer, base manual 2 days

grand total: 132 days spend to bring 'Gradual b/w' to Intel.

now, these numbers are very high. obviously, we can't recoup this effort from a single plug-in (especially since that one is free). we will spread this effort over all our plug-ins. adding this base, we estimate that each additional port will cost about two to three days of effort to port, test, and document. some more effort will be invested into our web site. but in the end, each plug-in will have cost about 3 days to port. that is a lot for just keeping the status quo, and our main reason to raise the price per plug-in. still, we are proud that we were able to pull it off, and are looking forward to releasing the new plug-ins. on monday. this sunday will be a *full* day off.

]]> Sat, 17 Mar 2007 21:13:12 +0100

it was a difficult question, and one we took a long time to answer: how do we price the upgraded (universal) plug-ins? keep the price? increase it? and if we increase it to cover for the additional amount of work that has gone into them, are we going to charge past customers for the upgrade? or perhaps have them re-purchase all their plug-ins because we think that the changes are great enough to warrant this?

well, here is what we have decided upon:
- although significant changes occurred, we do not believe in double-charging our customers. we are not 'big software' but 'fun software' for artists.
- we will increase the price of each plug-in that is upgraded to universal.
- people who have purchased a plug-in can upgrade for free. no upgrade fee, no re-purchase
- people who purchase a plug-in after it has been upgraded will have to pay the new price

that way we believe we can reward customers who already have purchased from us. thank you for your trust, and for your support!

]]> Sat, 17 Mar 2007 20:44:07 +0100

our preparations paid off handsomely. ladies and gentlemen, we have a plug-in that runs on intel



as soon as we update the manual, test a bit more, and are satisfied we'll let you download the plug-in. it'll be free.

]]> Sat, 17 Mar 2007 20:36:13 +0100 step 4 (build a fully-functional XCode-hosted plug-in) complete!

well, it was a difficult week, but at the end we have something to show for it:



need we say more? the plug-in fully compiles in XCode, is bundle-based, and sports some interface enhancements:
- on-line documentation from within the plug-in
- NTSC, PAL, and HD live switching
- movable (as many, many people have requested)
- always-live preview

we have also decided to slightly raise the minimal requirements: you now need at least 1024x768 resolution to run the plug-in.

next stop: compile on intel. this will take a bit longer, as we have to cross-compile, and cross-test. switching between two environments, always re-compiling and re-testing on two architectures is going to be trying. and we expect to hit a few difficult road-blocks, as the Intel-processors are low-endian, and video processing will be affected by this.

but for now we'll be enjoying the week-end.

]]> Fri, 16 Mar 2007 19:39:33 +0100 Universal binaries inch closer, old bugs make a re-appearance.

first the good news: we have made some significant progress on the road to universal binaries. we have an empty XCode-hosted plug-in shell compiling, and are busy building the first test-bed plug-in. our road to universal looks a bit like this:

1. build X-Code hosted plug-in shell that compiles (complete)
2. re-build plug-in as a bundle-based plug-in (complete)
3. build bundle-based interface, getting rid of old resources (we are here)
4. build a fully-functional XCode-hosted plug-in
5. build the same plug-in on an intel-machine
6. debug completely
7. begin porting all other plug-ins based on previous experience

we are currently at step 3 (switching the code to bundle-based interface). it's slow progress, but we are getting there.

now for the bad news: It appears that Apple has managed to re-introduce an old bug into iMovie 6.0.3 that it solved previously: the preview-bug now once again rears it's ugly head. unless you disable the preview before you apply a plug-in, iMovie will crash. as you may remember, Apple fixed this with 6.0.1 - now it's back. and yes, the bug is inside apple's code somewhere. what a disappointment.

]]> Mon, 05 Mar 2007 21:12:25 +0100

Today is a great day indeed. At least for us. This morning we received the one piece of compiler technology that was still missing in order to enable us to produce universal plug-ins. At least we think we do. Since we do not use C or C++ but rather prefer 'old-school' high quality code written in Pascal we until today had no way of integrating our Pascal Compiler's (the excellent Free Pascal, or FPC) code into an XCode-generated project (our old plug-ins were compiled using Metrowerk's CodeWarrior development environment). One hold-up was that XCode behaves rather brain-dead when compiling source for different compilers, and never was able to correctly use FPC when C or C++ code was present. Of course, Apple's development kit only relies on C code, and the fact that the last updated was published anno domini 2003 did not help.

This morning, Erling Johansen, Pascal programmer extraordinaire emailed us a work-around that might enable us to finally mix Pascal and C++ code in a way that is useful.

If this works, we still have some distance to go; the new plug-ins are bundle-based, we have to carbonize the interface, and we are looking into moving some of the code to CoreGraphics in order to alleviate the Carbon/Cocoa problems we have seen. This will be a lengthy process, and we do not expect to have universal plug-ins in a meaningful way ready before autumn. But at least one of the major stumbling blocks may have been removed. On the plus side we have most of our effect core, plus a lot of CoderGraphics-related code already moved to XCode.

We'll keep you posted.

And thanks, Erling. You really made our day!

]]> Thu, 22 Feb 2007 18:45:41 +0100 Apple sacrifices compatibility in favor of progress. our plug-ins pay the price. after chromakey, all plug-ins that use memory appear to be in danger of being paved over by the road to Intel. 'mix/blend movies' and 'multiple movies' appear to have become unstable with recent system releases.

as many of our customers know, the road to Intel compatibility is not half as smooth as some people (and Apple in particular) would have you to believe. in order to get OSX running on Intel and PowerPC simultaneously (a tremendous feat, and one that Apple deserves a lot of credit for), Apple had to sacrifice something. when Apple made the switch to OSX a few years ago, they devised a technology they called 'Carbon' that enabled people to harness the power of the new OS while remaining compatible with older (OS 8 and OS 9-based) applications. iMovie was one of those applications that originated in OS9 and acquired Carbon technology for OSX. the plug-in architecture for iMovie is still Carbon-based.

pushing new technologies, Apple forged ahead, and with each new OS release introduced new and enhanced features. until a few years ago, all applications relied on Apple's 'QuickDraw' technology to draw anything. to manage video and audio stream, Apple also developed the world-class QuickTime technology. then, a new drawing technology was introduced: 'Quartz'. it is an awesome technology, and very advanced. Quartz offers so many advantages that Apple has shifted QuickTime from QuickDraw as underlying drawing technology to Quartz. this is of course understandable, as it makes QuickTime a better product. if it weren't for a small snag: memory management.

you see, with OSX Apple introduced yet another new technology: Cocoa (actually, Cocoa was only new to Apple -- it has been around NeXT for quite a while). Cocoa is likewise an impressive technology that sports a feature that is as awesome as it is dangerous: automatic memory management (or better: 'garbage collection' as the compiler-savvy people call it). Cocoa can detect if a program isn't using a particular block of memory any more, and free it for other purposes.
this is great (especially for people programming in C, since most C programmers wouldn't know clean code if it backed over their head in a truck) as long as the application is programmed so that all memory is allocated exclusively using Cocoa technology.

iMovie in it's newer incarnations (definitely iMovie 5, 6) are Cocoa-based. yet, since iMovie demands that all plug-ins use Carbon-based code, the plug-ins rely on old-style memory management. many times this is no problem. most of our plug-ins receive a chunk of memory from iMovie, fiddle with it, and hand it back. these plug-ins have remained rock solid.
others have fared not as good. memory-intensive plug-ins (our chromakey line of plug-ins) were the first to hit this problem. what happens is this: the plug-in manages a lot of memory in addition to that handed to it by iMovie. this memory must be allocated, and the plug-in utilizes Carbon's (not Cocoa's) memory allocation routines. as long as the code does not leave plug-in, or you have lots of memory to spare, this can still work. however, when the plug-in has allocated lots of memory and the plug-in relinquishes control to iMovie, Cocoa kicks back into action. and it may happen that it sees a block of memory it knows nothing of, and re-allocates it to something else. this is what we some people a memory management collision. you call it a crashing bug. what we call it we'd rather not put into writing. in a nutshell, this is why chromakey still works in some situation, and crashes in others.

now, however, the situation is becoming worse. as apple pushes ahead, they have taken the (quite logical) step, and also brought large chunks of QuickTime over to Cocoa and it's new memory manager. with each new OS release, more and more pieces are moved to new technology. although this progress is nice in many ways, it spells disaster for some of our plug-ins. for example, our 'mix/blend' and 'multiple movie' plug-ins utilize QuickTime. up until recently, this was safe. now, suddenly, QuickTime relies on Cocoa. each time we call QuickTime in our plug-in, we leave the confines of Carbon, and re-enter Cocoa-managed territory. since QuickTime manages a lot of memory (most videos are huge), this is a lot like dancing in a dense mine field. sooner rather than later, QuickTime will de-allocate an unknown (Carbon-owned) block of memory, and and take dive off the deep end.

that is the reason why with each progressive system update we have seen more and more of our sophisticated plug-ins become more and more unstable. now, it appears that we have to issue warnings with regard to 'mix/blend movies' and 'multiple movies'. since we can see the logical progress, anything that loads large chunks of memory will eventually be hit by this. this will include all our plug-ins that load external pictures and/or movies. be warned that with newer systems 'picture in clip', 'multiple picture', and 'simple mask' may become unreliable as well.

all in all it appears that in the case of our plug-ins, trouble is the coinage for progress, and we have to pay it. it doesn't mean we have to like it. here's to the slight hope that maybe it's worth it.

]]> Mon, 01 May 2006 16:08:54 +0200 someone hacked our servers to distribute torrents of pirated material; we must pick up the tab.

or: why heinlein is right

the net is a tough place to live in -- even more so when you try to make a living there. it is also a playing ground for a lot of unsavory characters that - given half a chance - will take anyone for a ride. it appears that we gave those people more than half a chance. the damage done to us thankfully doesn't appear to be fatal, but we do feel it's impact.

here is what happened -- but first, some (perhaps irrelevant or even boring) information about the way we work: we do all our business using the internet. we use it as an enabling factor for some services, and a facilitator for others. our web site is maintained at what we affectionately call "the barn" - our offices. they are connected to the internet through a broadband connection; yet we do not host the site itself, as we deemed this to be too dangerous. should someone break into our production (or even worse: development) environment, we'd go out like a candle. so, rather than hosting our site ourselves, we turned to one of our country's biggest, and most experienced hosting provider. we synchronize our production machines regularly with the host, and keep this once-off setup for security reasons. a special process ensures that no production data is ever left on the host. we figured that this should keep us relatively safe. there simply are too many shady characters on the internet that know much more about security than we do. by separating our web facing server side from our mission-critical production side we thought that we were safe, and nothing bad could happen to us that we couldn't fix with a simple re-synch. the web-facing side was secure simply because it's being run by pros.

it turns our that although we were right about the first half of our reasoning (our production machines are still relatively safe), our hosting side wasn't. worse, a simple re-synch did not fix the problem; real damage was done. here's how: a few days ago we received a routine warning from our provider that our allocated disk quota for the web host was exceeded. this rather surprised us, as we know that our maximum disk quota is a factor of 10 above our normal use. we only pay for the space we actually use, but have a reserve in case we want or need additional space (this happens when we update web designs, as we duplicate the whole web content, and then test it before switching it on-line). we also use this reserve to test features of new products.

another reason for the reserve disk space is that we use it for file transfers to and from our 'pro clients' (i.e. those we work for on a contractual basis). for example, one client sends us a film clip to process. they upload it to the hosted web server, and we then download it to our production servers. since many of our customers (artists, usually) are even less tech-savvy than we are, we simplified the process for uploading data for them. since these clips can be rather large, we require a high reserve quota. since we have lots of download from both clients and internet users getting iMovie plug-ins, we also require a high bandwidth internet link.

  


and this is where we got hit. suddenly, in a corner of our web server, we detected a new folder containing gigabytes of unknown data. data that turned out to be pirated, copyrighted software. worse, there were lots of mysterious small files that were soon identified as torrent seeds. someone was illegally using our server to distribute stolen data. not only were they ripping off the copyright owners (as software developer/content provider ourselves we feel strongly about these things) - no, they were also leaving us holding the bag to pay for storage, and transmission costs. since our hosted server intentionally uses a high-bandwidth link to the internet backbone, transmission cost can become substantial. in the case of these torrents, they were. initial assessment is that the cost for the additional services will lose us about one month of profits. i guess we were lucky that we detected the break in relatively early on (only a few days had passed since the break-in). had this been going on for a few weeks, the resulting cost could well have killed us as a company.

exploiting a flaw in the hosting provider's OS (Unix), these crooks escalated their access privileges, disallowing us to remove the pirated files ourselves. as i write this, our provider is restoring the server. we really hope the bill for that won't be too high (they'll hopefully simply nuke the server, re-install, and then tell us to re-synch). we do not know if we'll get into hot water because our server was hijacked and used to illegally distribute copyrighted material. we *really* hope we don't, as we are ill prepared to fight a legal battle over this.

so, is there an upside to all this? perhaps - although only a small one: we learned a few things. 
firstly, i guess we where a bit naive and arrogant believing that we where safe. we thought that this would not happen to us. we only looked at our side of the problem, forgetting the (hosted) web-facing side. we also thought that since we are a small developer, with only a tiny web presence, no-one would be interested in us. we were proven wrong. nobody on the web, it seems, is too small or too uninteresting not to take advantage of. for these people, no low is too low. "drive someone out of business? i don't care - as long as I can torrent my software."

next up - an obvious lesson: read your provider's contracts. if you have a leverage factor attached to cost (in our case disk usage and bandwidth), make sure that you monitor it. we did not watch it closely enough, and got hit. we entered into those contracts fully knowing what they meant. those contracts were (and still are) good, and we will fulfill them. we will, however, take precautions that in the future we detect sudden changes in disk usage or bandwidth more rapidly. our provider was quite nice about this, has gone a few extra miles, and even indicated a willingness to re-negotiate some terms of our contract to help in similar situations.

the episode also drives home another point: as long as you are connected to the net, you are constantly threatened by miscreants. we'll have to beef up our security, and probably regularly audit our security measures. how much this will cost we don't know. but we simply can't afford a few more break-ins like this. it also shows how the shadowy so-called 'sub-culture' of illegal file sharing has tangible effects on small companies. to quote Heinlein: "there ain't no such thing as a free lunch." in addition to the obvious theft of copyrighted material, bandwidth and storage also have to be paid for. most illegal torrents nowadays exploit, and maybe even ruin, some unwitting small company like us that did or could not invest enough into it's security.

we also learned something nice, though: that we can depend upon our clients. you see, because of all this we are also in a tight spot now that we can't easily allow our clients to transfer data any more. we will surely find a new way, but that will take time, and effort. luckily, most of our clients understand the predicament we are in, and are willing to jump through a few more hoops to work with us. for this we thank them and you.

]]> Wed, 05 Apr 2006 18:17:06 +0200