HDV File Corruption Issues

OK so before on my dailies I've been downconverting the footage direct from the cam to the G5. Using FCP 5.0.1 and until now never had an issue.

I obviously prefer to have the stock in HDV so now I've completely cleared out the hard drives of the 260Gb of SD stock and am in the process to recapture the HDV stock. Problem is when importing the rendered clip / archive file, they play back completely corrupted, looks more like a green laser show with green lines almost rhythmically pulsing across the screen. This happens on some clips and other not, some from the same master tape, others completely un-related. This is really frustrating. I would never have noticed but I am in a final edit for a big project and I'm obviously checking through differing files for inclusion.

This is really starting to underline my dislike for the HDV format as a whole. Great images are possible but the issues arising from the compression cause more work flow problems than the format solves......advancing technology??

Anyone else having this, experienced it or remedied it? I'm exporting Quick Time files , no audio.

Cheers,

Mark.

Mark, I'm confused... are you getting corrupted files capturing or exporting?

It would also help to give us OS and computer info.

Hi Drew,

I'm running a less than two year old PowerPC Mac G5 (2Ghz) with 2 GB DDR SDRAM - Mac OS X version 10.4.6 - Space and storage is not an issue as I have over 1 TB of storage available.

FCP Version 5.4.1

Basically the clip plays fine on playback from the deck / cam. When I import to FCP on the preview pane I can see the interference. When I then re-open the clip either on the time line or from its storage location from within the computer the interference is very apparent. I guess there must be some problem with the chain of command from tape output and capture. I've tried swapping out Fire Wires, no apparent change.

Very frustrating. I'm capturing HDV so no down conversion instructions have been programed to the camera.

Any light shed on to this would be welcomed.

Cheers,

Mark.

Mark are you just sucking it in as m2t or is FCP converting it to it's own special HDV codec?

Wags,

with FCP it automatically creates QuickTime files using that proprietary codec. There have been issues with QT messing with Gamma curves of imported imagery but I can't say I've noticed any massive problems there.

On another note just to show how quirky (word of the day) this format can be. I recaptured the same footage again, first thing this morning. No problems whatsoever. Could it be that the artifacts are more prominent with a hot machine!! I had been capturing some 4 hours yesterday!! Maybe temp has a negative impact on the compression data when capturing.......anything is possible.

Just a pain in the butt to think that we have to check every second of every ingestion for corrupt files / dropped frames etc. I always prefer to set to capture and then bugger off and do other related stuff for the dailies.....prepping photo slide shows of the day or whatever.

Any further ideas?

Cheers,

Mark.

Arrrr then if it's converting it on the fly as it comes in then it would be one of two things.

I can have the same problem here if my drives are nearly full or they are in need of defraging.

You can defrag or simply move all the stuff from one drive to another as this is the same as defraging. Stacks all the data nice and neat again on the drive.

Another reason is if the processor cannot keep up, make sure you have no other programs running. Can you see your CPU % as it comes in? If it is overheating as well everything will slow down.

Wags,

I'm capturing to a brand new 300 GB scratch disc, a Maxtor 400 / USB 2.0 (16MB Cache / 7200rpm) There is currently 175GB still available on the disc.

I'm assigning the footage once captured and trimmed (not that this is anything to do with the ingestion process) in FCP to a second hard drive, identical to the scratch drive.

No other programs are running whilst I capture. So I guess I can only limit myself to a few hours capture a day?? Bit of a bummer seeing as I've got a massive edit on that needs all the footage available. I've logged tapes too so it will just be a time consuming effort to just select the exact tapes I need. Ahhh the joys of our craft.

I'm seeing if I can find the performance monitor when I'm capturing. I think its in Utilities somewhere, I think I have Tech Tools Pro so will look into that.

Cheers, any more feedback from peeps would be appreciated too.

Best,

Mark.

Mark

There are many things that can cause this. A step by step elimination process is necessary so it doesn't repeat itself.

1. Are you capturing with the firewire drives daisy chained with your camera? That usually kills the bandwidth of the the firewire bus.

2. Update your software to the latest versions. You say you have version 5.0.1 then 5.1.4. Make sure you have the updated versions of FCP AND quicktime, along with any Pro Apps updates.

3. What are your FCP capture and control settings? Are you batch capturing?

4. Heat doesn't hurt the computer as much as it does the drives. To see what is going on, set up your Activity Monitor to observe your disk and CPU activity. It would also help to kill off other processes and programs.

Unfortunately, troubleshooting means going into "moron" mode and checking your steps. It's very easy to overlook a setting and/or update.

Also USB 2 is not recommended for camera capture. You problem may be right there.

Drew,

Thank's for taking the time with this. OK, I'm gonna dump all I have thus far captured which is not that much, maybe around 12oGB on to a LaCie 500GB drive. Then I'll plug that directly to the G5 and set scratch to that. Hoping there will be a marked difference in performance.

Answers to your questions:

i) Yes, they each of the Maxtors have two firewire ports so I have them daisy chained (scratch is set at in the immediately connected drive) that in turn is then connected to a second and third in the chain is the LaCie.

ii) OK, but will do the tests with the LaCie before I start spending cash.

iii) Not batch capturing to reduce wear on the heads of the camera. Straight run through of the tape, ingest the lot, clip the archive files render to the second drive and then ditch the captured tape file. Keep thew master tape in master archive.

iv) Nothing else is running in the background. Just FCP and capturing. I have a second G5 for all the other bits I need to do for the dailies etc.

Thanks for the tips. Will post results of the LaCie drive.

Thanks again,

Mark.

I was sent a USB 100GIG drive the other day and I copied a 70 GIG consolidated HDV project to it.

I then opened the project on it and scrubbed the timeline, it was not as responsive as my SATA drives in the computer.

Mark

If you have your camera on the same firewire daisychain as your firewire drives, that is also something you need to change. Firewire400 (1394a) has bandwith of 400mbps (theoretical), which translates to about 50MB/s. Realistically it's about 30MB/s. HDV requires 4MB/s at most but if you capture on the same bus, the I/O bandwidth strain is problematic. Every manufacturer suggests not to use the camera and drives on the same bus for capture.

You don't have to spend cash to update your software. Just download what the latest updates are.

Check your FCP settings as well.

Hi Guys,

My cam is not connected in the same hard drive daisy chain. There are two fire wire ports in the back of the G5. One for the camera and the other is set to the initial (scratch) drive. The others drives run from the second fire wire out port from that first hard drive in the chain.

I'm checking all settings. Half way through the transfer to the LaCie too.

Cheers,

Mark.

Mark

The 2 ports may run on the same bus so essentially you have it all in one bus, which could be your problem. I would suggest you unplug the other drives and use one for capture. Since this is intermittent, if you've set FCP to capture to 2 different drives, it would cause the OS to look for the 2nd drive and cause corruption. The best way to do it is to get a FW800-FW400 converter wire and run the drives off that. Or capture to your internal hard disk. FW drives are a bad idea for capture and archiving long term.

Drew I was thinking just that..same bus..

Another option is to just capture the m2t (15GIGS per hour) and then once you have a heap of files in a watch folder, fire up your encoder and it will convert them overnight to the QT codec. Not sure if you can do that on a MAC. That way you are guaranteed a clean conversion.

I can with Procoder Express, m2t to Canopus HQ.

Well I'll be a Monkeys Uncle,

Just goes to show we live and learn. OK, I've got a handle on it. I'll be re wiring and doing a complete reconfiguring tomorrow. All sounds good.

Guys, thanks for the time spent shedding light on this. I appreciate it.

Cheers,

Mark.

As the others have said I've had problems in the past daisy chaining a camera onto an external FW drive as opposed to having them on 2 separate busses.

Can you run Windows on your Mac? If so and you're still having trouble with FCP capture then you might try capturing raw .m2t with HDVSplit, which is what I use. It makes a nice job and the flexible automatic file naming is great. Note it does give loads of "some packets lost" errors but the files are still useable. I think the errors are because of broken timecode on the tape caused by me having the Z1 in "quick record" mode, but the result is not too serious, e.g. the last half second or so of the clip has no audio and frozen video but can just be edited out.

Hey Guys,

OK, so the result is that I've reverted to SD. Call me a wimp, call me a wuss but I'm on a deadline for this edit, I've cleared things with the client and seeing as this initial version is for Internet broadcast there is no real rush for the HDV edit. I have kept the base layer of the HDV string pre effects etc in case I can sort this machine out.

I think at the end of the day I just need to upgrade the system I am using at the moment. Mind you its only about a year old. They don't make 'em like they used to eh, huh". So I recaptured everything again to the internal hard drive set as scratch. I had transfered all the data I had, artwork, designs and general office stuff, to one of the Matrox drives fom a LaCie 500GB. I now have the LaCie as a video storage drive connected to the second firewire port of the G5 and the Matrox Drives are connected via USB 2 to the same computer. I'm not planning on using them for video storage.

So after recapture I started adding some effects to the HDV material and the same dropped frame, green laser show across the footage reappeared. Almost rhythmical in appearance it would stop on footage that had no effects sourced to it. I'm also running SafeRT but every 3 seconds or so I get the warning dialog come up for dropped frames. All my settings are set for HDV1080i60 Audio is set at 48kHz as per the accompanying track issued by the client. I understood from day 1 that HDV would be a render intensive format and I find myself waiting massive amounts of time to be able to check effect motion results.

Such is life, ideas anyone?

Cheers,

Mark.

So you are saying the raw footage on the drives is okay now, just when you add effects to it you get problems?

Mark I capture HDV with a 3 year old G4 powerbook on the internal drive with no problems. It's not your system speed.

First of all, you have to be clear.

1. Is your captured clips affected only?

2. Or is it the rendering?

3. After capture, convert to Apple Intermediate COdec.

4. Make sure you have all the updated software including quickitime.

If it is now only rendering that causes it, then do this to check if it is just during playback.

Set the playback monitor to firewire and watch it on the camera...if it's doing it there then you have a software issue. I'd uninstall FCP and reinstall and update everything.

Hi Guys,

Wags - Thats about the size of the problem yeah. The clips capture and play fine once I have done the 'Capture Now' operation. I can store them on my hard drive, play them straight from the drives and no problem. Once I start sourcing effects to the clips thats when I get the corruption. Really frustrating as you can imagine to sit through a 2 hour render and then have dropped frames and frames that are littered with artifacts etc.

Drew - I'm starting from scratch to load in clips to hard drives. At the moment I'm not experiencing problems capturing or play back of those same clips, see above. I believe it could have something to do with the rendering maybe? Not that quick on the old tech side of computers mate. Its my biggest bind to have to work with computers at all but there ya have it, one day I'll have my own editor!! Until then I gotta struggle through this.

Are you also saying that it is possible to swap the codec for editing once capture has been effected? I always thought it stays on the codec for the format as shown on the Audio / Video Preferences. Shows how much I know.

Also, whilst we're at it. I'm looking to get a 23fps look to some stuff here. How do I program the timeline for that or would it be better to rely on the Strobe filter?

Cheers,

Mark.