Automatic Activation, Hotplug and UDEV, Configuration
Post Reply
shaman
Posts: 2
Joined: 03 Jun 2014, 01:12

Possible race condition during mode switch

Post by shaman » 03 Jun 2014, 03:03

In developing some new software I ran into an issue with usb_modeswitch. The best I have been able to determine is that there is a race condition.

Setup:
Arch Linux running on a Raspberry Pi
Linux Kernel - I have tried both 3.6.11-11 and 3.12.20-3 (which is the latest)
I am using a Verzion USB760 modems

Problem:
During system boot the modem identifies two mass storage devices: a drive and a cdrom.
As the corresponding drivers are initializing the udev rules detect the device availability
and trigger the modeswitch. This is were the fun begins... sometimes the modeswitch completes
but the 4 ttyUSBX devices do not show up and short of power cycling the Pi nothing you do fixes
the problem. The Pi and the modem must power cycle in order to straight out the problem.

What I have noticed is that if you slow down the modeswitch give the kernel time to complete
the driver initialization before the modeswitch is activated then it works flawlessly but if it
happens to quickly then it can leave the modem/kernel is a weird state that will not easily
recover from (and usually requires a power cycle to fix).

I have a solution now which does not use modeswitch at all, I was able to determine a way to resolve the race condition reliably, I also tried setting "SetStorageDelay=20" for some reason this did not help it only slowed down the process of doing a modeswitch.

My current solution I wait for the /dev/sr0 device to appear at which point I use 'eject /dev/sr0' to active the modeswitch and then everything works great (never fails).

Any suggestion or insight into what is going on would be helpfull.

LOM
Posts: 1404
Joined: 11 Jul 2012, 15:14
Location: Koh Samui, TH

Re: Possible race condition during mode switch

Post by LOM » 03 Jun 2014, 05:51

usb_modeswitch basically does the same as the linux eject cmd, both of them sends the same usb bulk msg "Stop unit" aka "Eject media".

I assume you are using an old version of usb_modeswitch which could fail to eject if the device was busy, for instance when the storage driver was reading the long chain of scsi attributes from the storage devices.

shaman
Posts: 2
Joined: 03 Jun 2014, 01:12

Re: Possible race condition during mode switch

Post by shaman » 04 Jun 2014, 20:55

I was pretty sure that modeswitch was doing an eject.

I am using the latest version of usb_modeswitch.

I am new to debuging a USB device so I had to do a little guessing and through observation I was able to see some paterns. From what I could tell it appeared that the Linux kernel drivers were in the middle of initializing the mass storage devices when the "Stop unit" message was sent this caused the hardware to denumerate and then re-enumerate in the switched mode. The drivers then seemed to get confused and the modem would then also get confused and then either the 4 ttys would not enumerate or they would enumerate but they would not work.

If I slowed down the process enough then it would work every time. The trick was the make sure that the drivers had completed initialization before sending 'Stop unit', my solution was to wait for the '/dev/sr0' device to show up and then send the 'Stop unit'. I tried to configure usb_modeswitch to have longer delays but for some reason that did not help. It appeared that the udev rule would trigger on the wrong event which caused it to trigger to early. Most of the time it worked but sometimes it would not and once screwed up it was near impossible to fix using software only (especially since a RaspberryPi does not power cycle USB devices when the system is rebooted).

In an attempt to recover the modem after it wigged out I did figure out how to issue a USB_RESET to individual usb devices to see if that would help. The problem though was that the Verizon modem does not reset the entire device it only resets the USB chip.

Post Reply