Import digital video

From Finninday
Jump to: navigation, search

I'm having difficulty importing digital video from a Canon Elura 40MC digital video camcorder NTSC.

It faithfully recorded about 10 60-minute DVM cassettes of travel-related footage, but I haven't been able to get the video into a computer to do any editing or sharing.

options tried

kino

Doesn't recognize the camera when connected by firewire.

much time goes by since last attempt

Starting this project again from scratch.

  • Located the firewire cable that has a plug that fits into the Elura.
  • Located the batteries and charger for the Elura.
  • Located the box of untransfered video.

While I wait for the batteries to charge, I can form a strategy. This attempt will be different, because I think I actually had some success at one point but didn't document it.

There is a directory with a few files in it. The first three tapes have been turned into 13GB AVI files. Each is about 1 hour long, 720x480, 30 frames per second, 32GHz sample rate audio.

Playing these files with VLC generates lots of loud garbled audio, but correct video. Playing with Xine, there is no audio, but the video looks right.

Also in the directory are some files generated by iMovie that are DV instead of AVI. They have good video and audio that you can understand, but still there are lots of sound artifacts.

I'm pretty sure that I want to try VLC as the software to open the video camera device and record. But I'm not sure how to pick the right container, video codec, and audio codec.

Another option might be handbrake, but that isn't installed yet on the one firewire enabled server.

Maybe Kino is worth another shot. It is now at version 1.3.4 on the server.

Also dvgrab looks promising (https://wiki.ubuntu.com/UbuntuStudio/PackageList).

dvgrab

rday@weasel:/drobo2/home-movies/tmp$ dvgrab
rom1394_0 warning: read failed: 0x0000fffff0000414
error reading config rom directory for node 0
rom1394_1 warning: read failed: 0x0000fffff0000414
error reading config rom directory for node 1
Found AV/C device with GUID 0x00008500005e42e9
Waiting for DV...
Capture Started
^C"dvgrab-001.dv":   215.95 MiB 1887 frames timecode 00:03:03.03 date 2003.07.29 22:47:48
Capture Stopped

rday@weasel:/drobo2/home-movies/tmp$ file *
dvgrab-001.dv: DIF (DV) movie file (NTSC)
rday@weasel:/drobo2/home-movies/tmp$ ls -l
total 221356
-rw-r--r-- 1 rday rday 226440000 Oct 24 23:57 dvgrab-001.dv

Result has garbled audio.

rday@weasel:/drobo2/home-movies/tmp$ dvgrab -f dv2
rom1394_0 warning: read failed: 0x0000fffff0000414
error reading config rom directory for node 0
rom1394_1 warning: read failed: 0x0000fffff0000414
error reading config rom directory for node 1
Found AV/C device with GUID 0x00008500005e42e9
Waiting for DV...
Capture Started
# no audio
^C"dvgrab-001.avi":    36.27 MiB 306 frames timecode 00:00:00.00 date 2013.10.25 00:02:19
Capture Stopped


Result has garbled audio.

Tried the experiment of saving in each of these formats: dv1, dv2, avi, raw, qt, mov. Each time the audio was garbled for the first five seconds, and then absent for the rest of the sample. Still need to try jpeg, jpg, mpeg2, hdv.

mpeg2 fails even more:

rday@weasel:/rootback/home-movies$ dvgrab -f mpeg2
rom1394_0 warning: read failed: 0x0000fffff0000414
error reading config rom directory for node 0
rom1394_1 warning: read failed: 0x0000fffff0000414
error reading config rom directory for node 1
Found AV/C device with GUID 0x00008500005e42e9
Waiting for HDV...
^C""     0.00 MiB 0 frames
Capture Stopped
Error: no HDV. Try again before giving up.

Same with hdv format.

rday@weasel:/rootback/home-movies$ dvgrab -f hdv
rom1394_0 warning: read failed: 0x0000fffff0000414
error reading config rom directory for node 0
rom1394_1 warning: read failed: 0x0000fffff0000414
error reading config rom directory for node 1
Found AV/C device with GUID 0x00008500005e42e9
Waiting for HDV...
^C""     0.00 MiB 0 frames
Capture Stopped
Error: no HDV. Try again before giving up.

Exporting to jpg format works, but actually makes a pile of jpgs and drops the audio. So I'll count that as not working.

more time goes by

usb port?

  • I think the last time I tried, I established that there is a problem with the firewire subsystem. But the video camera also has a usb port. I don't think I've tried the usb port.
  • on an Ubuntu 14.04 system, hot-plugging the usb port shows nothing in /var/log/syslog or dmesg

try firewire again

  • openshot is installed. does it work?

look at device connection messages

I already have one firewire device in play which is the drobo.

When I have the camcorder on and in play mode, and then connect to the firewire bus, I see this in syslog:

root@weasel:/var/log# tail -f syslog | grep firewire
Sep  5 10:41:29 weasel kernel: [320276.022176] firewire_core 0000:01:08.0: phy config: new root=ffc2, gap_count=7
Sep  5 10:41:29 weasel kernel: [320276.022718] firewire_ohci 0000:01:08.0: node ID not valid, new bus reset in progress
Sep  5 10:41:29 weasel kernel: [320276.023198] firewire_ohci 0000:01:08.0: node ID not valid, new bus reset in progress
Sep  5 10:41:29 weasel kernel: [320276.520063] firewire_core 0000:01:08.0: giving up on node ffc1: reading config rom failed: bus reset
Sep  5 10:41:29 weasel kernel: [320276.520120] firewire_core 0000:01:08.0: rediscovered device fw4
Sep  5 10:41:29 weasel kernel: [320276.523969] firewire_core 0000:01:08.0: rediscovered device fw3
Sep  5 10:41:31 weasel kernel: [320278.020076] firewire_sbp2 fw3.0: ORB reply timed out, rcode 0x11
Sep  5 10:41:31 weasel kernel: [320278.020104] firewire_sbp2 fw3.0: ORB reply timed out, rcode 0x11
Sep  5 10:41:31 weasel kernel: [320278.220797] firewire_sbp2 fw3.0: reconnected to LUN 0000 (0 retries)
Sep  5 10:41:33 weasel kernel: [320280.220071] firewire_sbp2 fw3.0: ORB reply timed out, rcode 0x11
Sep  5 10:41:33 weasel kernel: [320280.220090] firewire_sbp2 fw3.0: failed to reconnect
Sep  5 10:41:33 weasel kernel: [320280.420523] firewire_sbp2 fw3.0: error status: 0:10
Sep  5 10:41:33 weasel kernel: [320280.421304] firewire_sbp2 fw3.0: logged in to LUN 0001 (0 retries)
Sep  5 10:41:34 weasel kernel: [320281.544900] firewire_core 0000:01:08.0: created device fw2: GUID 00008500005e42e9, S100
Sep  5 10:41:34 weasel kernel: [320281.545173] firewire_core 0000:01:08.0: phy config: new root=ffc2, gap_count=7
Sep  5 10:41:34 weasel kernel: [320281.545516] firewire_ohci 0000:01:08.0: node ID not valid, new bus reset in progress
Sep  5 10:41:35 weasel kernel: [320282.048113] firewire_core 0000:01:08.0: rediscovered device fw4
Sep  5 10:41:35 weasel kernel: [320282.064825] firewire_core 0000:01:08.0: rediscovered device fw2
Sep  5 10:41:35 weasel kernel: [320282.064846] firewire_core 0000:01:08.0: phy config: new root=ffc2, gap_count=7
Sep  5 10:41:35 weasel kernel: [320282.065184] firewire_ohci 0000:01:08.0: node ID not valid, new bus reset in progress
Sep  5 10:41:35 weasel kernel: [320282.065591] firewire_ohci 0000:01:08.0: node ID not valid, new bus reset in progress
Sep  5 10:41:35 weasel kernel: [320282.066003] firewire_ohci 0000:01:08.0: node ID not valid, new bus reset in progress
Sep  5 10:41:35 weasel kernel: [320282.066418] firewire_ohci 0000:01:08.0: node ID not valid, new bus reset in progress
Sep  5 10:41:35 weasel kernel: [320282.564107] firewire_core 0000:01:08.0: rediscovered device fw4
Sep  5 10:41:35 weasel kernel: [320282.580768] firewire_core 0000:01:08.0: rediscovered device fw2
Sep  5 10:41:35 weasel kernel: [320282.580786] firewire_core 0000:01:08.0: phy config: new root=ffc2, gap_count=7
Sep  5 10:41:35 weasel kernel: [320282.581124] firewire_ohci 0000:01:08.0: node ID not valid, new bus reset in progress
Sep  5 10:41:35 weasel kernel: [320282.581532] firewire_ohci 0000:01:08.0: node ID not valid, new bus reset in progress
Sep  5 10:41:36 weasel kernel: [320283.080101] firewire_core 0000:01:08.0: rediscovered device fw4
Sep  5 10:41:36 weasel kernel: [320283.096690] firewire_core 0000:01:08.0: rediscovered device fw2
Sep  5 10:41:36 weasel kernel: [320283.096705] firewire_core 0000:01:08.0: phy config: new root=ffc2, gap_count=7
Sep  5 10:41:38 weasel kernel: [320285.056090] firewire_core 0000:01:08.0: giving up on node ffc0: reading config rom failed: bus reset
Sep  5 10:41:38 weasel kernel: [320285.568475] firewire_core 0000:01:08.0: giving up on node ffc0: reading config rom failed: bus reset
Sep  5 10:41:38 weasel kernel: [320285.684258] firewire_sbp2 fw3.0: released target 7:0:0

Looking at /dev/fw* while the camcorder is connected:

root@weasel:/dev# ls -l fw*
crw-------  1 root root  249, 0 Sep  1 17:43 fw0
crw-------  1 root root  249, 1 Sep  1 17:43 fw1
crw-rw----+ 1 root video 249, 2 Sep  5 10:41 fw2
crw-------  1 root root  249, 3 Sep  5 10:41 fw3
crw-------  1 root root  249, 4 Sep  5 10:40 fw4

video devices do not exist:

root@weasel:/dev# ls -l /dev/video*
ls: cannot access /dev/video*: No such file or directory

dvgrab works. no garbled audio

I tried saving to a local filesystem instead of a slow remote filesystem and got good results:

rday@weasel:~/home-movies$ dvgrab
rom1394_0 warning: read failed: 0x0000fffff0000414
error reading config rom directory for node 0
rom1394_1 warning: read failed: 0x0000fffff0000414
error reading config rom directory for node 1
Found AV/C device with GUID 0x00008500005e42e9
Waiting for DV...
Capture Started
^C"dvgrab-001.dv":   163.31 MiB 1427 frames timecode 00:02:44.27 date 2002.09.12 15:59:13
Capture Stopped

Copied from weasel to a fedora laptop and played with vlc and had clear audio and video. Wow.

But after switching to a new tape, dvgrab thinks the camera is gone:

rday@weasel:~/home-movies$ dvgrab
rom1394_0 warning: read failed: 0x0000fffff0000414
error reading config rom directory for node 0
rom1394_1 warning: read failed: 0x0000fffff0000414
error reading config rom directory for node 1
send oops
send oops
send oops
Error: no camera exists

The thing I did differently that time was to launch dvgrab while the camera was not playing. dvgrab was able to trigger the camera to start playing, but had then couldn't see the camera and could not stop the playback.

troubleshooting "no camera found"

with the drobo and camera disconnected, the firewire devices look like this:

root@weasel:~/home-movies# ls -l /dev/fw*
crw------- 1 root root 249, 0 Sep  1 17:43 /dev/fw0
crw------- 1 root root 249, 1 Sep  1 17:43 /dev/fw1
crw------- 1 root root 249, 2 Sep  5 11:16 /dev/fw2
crw------- 1 root root 249, 4 Sep  5 10:40 /dev/fw4

after plugging in just the camera while powered on but not playing

root@weasel:~/home-movies# ls -l /dev/fw*
crw-------  1 root root  249, 0 Sep  1 17:43 /dev/fw0
crw-------  1 root root  249, 1 Sep  1 17:43 /dev/fw1
crw-------  1 root root  249, 2 Sep  5 11:16 /dev/fw2
crw-rw----+ 1 root video 249, 3 Sep  5 11:48 /dev/fw3
crw-------  1 root root  249, 4 Sep  5 10:40 /dev/fw4

Still dvgrab doesn't see a camera:

root@weasel:~/home-movies# dvgrab
rom1394_0 warning: read failed: 0x0000fffff0000414
error reading config rom directory for node 0
Error: no camera exists

More info on devices on firewire bus

root@weasel:/sys/bus/firewire/devices# ls -al
total 0
drwxr-xr-x 2 root root 0 Sep  5 11:54 .
drwxr-xr-x 4 root root 0 Sep  5 11:54 ..
lrwxrwxrwx 1 root root 0 Sep  5 11:54 fw0 -> ../../../devices/pci0000:00/0000:00:09.0/0000:01:08.0/fw0
lrwxrwxrwx 1 root root 0 Sep  5 11:54 fw1 -> ../../../devices/pci0000:00/0000:00:0b.0/0000:02:00.0/fw1
lrwxrwxrwx 1 root root 0 Sep  5 11:54 fw2 -> ../../../devices/pci0000:00/0000:00:09.0/0000:01:08.0/fw2
lrwxrwxrwx 1 root root 0 Sep  5 11:54 fw3 -> ../../../devices/pci0000:00/0000:00:09.0/0000:01:08.0/fw3
lrwxrwxrwx 1 root root 0 Sep  5 11:54 fw3.0 -> ../../../devices/pci0000:00/0000:00:09.0/0000:01:08.0/fw3/fw3.0
lrwxrwxrwx 1 root root 0 Sep  5 11:54 fw4 -> ../../../devices/pci0000:00/0000:00:09.0/0000:01:08.0/fw4

unplugging the camera and replugging it seems to have allowed the firewire bus to settle in a way that works. There are a lot of errors however and it takes a long time for the bus to settle. The errors look like this:

Sep  5 11:58:22 weasel kernel: [324889.746372] firewire_ohci 0000:01:08.0: node ID not valid, new bus reset in progress
Sep  5 11:58:22 weasel kernel: [324889.748765] firewire_core 0000:01:08.0: parent port inconsistency for node 0: parent_count=1
Sep  5 11:58:22 weasel kernel: [324889.748769] firewire_core 0000:01:08.0: topology build failed
Sep  5 11:58:23 weasel kernel: [324890.248121] firewire_core 0000:01:08.0: rediscovered device fw2
Sep  5 11:58:23 weasel kernel: [324890.302175] firewire_core 0000:01:08.0: rediscovered device fw3
Sep  5 11:58:23 weasel kernel: [324890.302198] firewire_core 0000:01:08.0: phy config: new root=ffc1, gap_count=5

That set of errors appears many times per second for a few minutes.

first tape invocation

When the camera is actually detected, the importing works pretty well with this simple invocation:

dvgrab -a

That parameter is dividing the files automatically when a break is detected.

One 60-minute tape is generating 12GB of dv files that look like this:

rday@weasel:~/home-movies/tape01$ file dvgrab-001.dv 
dvgrab-001.dv: DIF (DV) movie file (NTSC)

debugging steps I haven't tried yet

As root, run
# echo -1 > /sys/module/firewire_ohci/parameters/debug
Then plug in/ switch on the camcorder. This should normally produce lots of output in dmesg. 

lshw says this about firewire:

        *-firewire
             description: FireWire (IEEE 1394)
             product: uPD72873 [Firewarden] IEEE1394a OHCI 1.1 Link/2-port PHY Controller
             vendor: NEC Corporation
             physical id: 8
             bus info: pci@0000:01:08.0
             version: 01
             width: 32 bits
             clock: 33MHz
             capabilities: pm ohci bus_master cap_list
             configuration: driver=firewire_ohci latency=32 maxlatency=44 mingnt=20
             resources: irq:18 memory:fdeff000-fdefffff

        *-firewire
             description: FireWire (IEEE 1394)
             product: VT6315 Series Firewire Controller
             vendor: VIA Technologies, Inc.
             physical id: 0
             bus info: pci@0000:02:00.0
             version: 01
             width: 64 bits
             clock: 33MHz
             capabilities: pm msi pciexpress ohci bus_master cap_list
             configuration: driver=firewire_ohci latency=0
             resources: irq:17 memory:fddff000-fddff7ff ioport:9c00(size=256)

root@weasel:~/home-movies# lspci | grep 1394
01:08.0 FireWire (IEEE 1394): NEC Corporation uPD72873 [Firewarden] IEEE1394a OHCI 1.1 Link/2-port PHY Controller (rev 01)
02:00.0 FireWire (IEEE 1394): VIA Technologies, Inc. VT6315 Series Firewire Controller (rev 01)
lspci -v

01:08.0 FireWire (IEEE 1394): NEC Corporation uPD72873 [Firewarden] IEEE1394a OHCI 1.1 Link/2-port PHY Controller (rev 01) (prog
-if 10 [OHCI])
        Subsystem: NEC Corporation uPD72873 [Firewarden] IEEE1394a OHCI 1.1 Link/2-port PHY Controller
        Flags: bus master, medium devsel, latency 32, IRQ 18
        Memory at fdeff000 (32-bit, non-prefetchable) [size=4K]
        Capabilities: [60] Power Management version 2
        Kernel driver in use: firewire_ohci

02:00.0 FireWire (IEEE 1394): VIA Technologies, Inc. VT6315 Series Firewire Controller (rev 01) (prog-if 10 [OHCI])
        Subsystem: VIA Technologies, Inc. VT6315 Series Firewire Controller
        Flags: bus master, fast devsel, latency 0, IRQ 17
        Memory at fddff000 (64-bit, non-prefetchable) [size=2K]
        I/O ports at 9c00 [size=256]
        Capabilities: [50] Power Management version 3
        Capabilities: [80] MSI: Enable- Count=1/1 Maskable+ 64bit+
        Capabilities: [98] Express Endpoint, MSI 00
        Capabilities: [100] Advanced Error Reporting
        Capabilities: [130] Device Serial Number 00-11-06-ff-ff-00-01-44
        Kernel driver in use: firewire_ohci

I should really be using that 64-bit card instead of the 32-bit one.

How do I know which firewire controller is in use?

observations

It appears that the drobo and the video camera don't share the firewire bus very well. Trying to copy video from the camera while the camera and drobo are on the same bus has caused the drobo to crash. Trying to find the camera while the camera and drobo are on the same bus has caused the camera to be inaccessible.

I have tried moving both the drobo and the camera to the newer 64-bit firewire card, but now drobo is slower and tends to crash more. I'll try moving back to the old firewire card next to see if performance will come back or if I just need to remove the camera.