Bcdl And Usb - Some Thoughts And Requests, Anyone can reach Reanimatolog? Options

[jaclaz]

Since a few days have passed since this post of mine:
http://www.911cd.net/forums//index.php?s=&...st&p=121664
with no related reply, but it is well possible that it was not seen by any people as it is in the huge Dietmar's USB thread, I have decided to make a new one, possibly in a more suitable section of the forum, with some more info added to it.

BCDL appears to be able to load standard DOS CD-ROM drivers, including, in some occasions, USB ones, from the Help file coming with BCDL 1.50z:

v1.50Z
Supports more than one device (e.g. ATAPI), booting can be done from any device
TRIES to load the following drivers (if present on the floppy):
vide-cdd.sys (1)
usb_cd.sys (3)
usbaspi.sys + aspicd.sys (3)
aspi8u2.sys + aspicd.sys (4)
aspi8dos.sys + aspicd.sys (2)
aspi7dos.sys + aspicd.sys (4)
aspi4dos.sys + aspicd.sys (3)
aspi2dos.sys + aspicd.sys (3)

(1) - works almost always
(2) - works sometimes
(3) - did not work for the author, but may work for you
(4) - no information

I cannot see why the same code cannot load some other DOS drivers, as an example those collected here :
http://johnson.tmfc.net/dos/usbdrv.html
that cover most if not all USB devices.

BCDL.BIN appears to "somehow" extend the BIOS and, through the DOS driver VIDE-CDD.SYS, allows for booting a CD on machines which BIOS does not support it.

I already found, in my experiments with Grub4dos, that SBM (Smart Boot Manager) code does not boot ALL tested CD-Roms, whilst using the BCDL (I used version 2.01, but the basic idea should be the same) even those CD images were booted succesfully.

It is not difficult, to boot "problematic" CD's to add a "normal" or zipped SBM or BCDL floppy image to the Grub4DOS menu.lst.

Tinybit, the author of Grub4dos has added to latest releases, right now latest is 0.4.2pre10:
http://grub4dos.jot.com/WikiHome/grub_for_dos-0.4.2pre10.zip
the SBM code that is Open Source.

In this message:
http://www.911cd.net/forums//index.php?s=&...st&p=122606

But you may contribute to GRUB4DOS a USB driver, just like the cdrom driver that is currently builtin grub4dos.

and in some correspondence I had with him, Tinybit asked for a possible USB BIOS extension to be integrated in Grub4dos next release

Now, I see the method used by Reanimatolog (Alex Kopylov) a very interesting one, as it is modular and should be easily modifiable if any new and better DOS driver comes out.

Problem is that though Freeware, and with a remarkably "open" license:

Bootable CD Wizard v2.0a1
Copyright ©2004 by Alex Kopylov
All rights reserved.

Permission is hereby granted, free of charge, to any person
obtaining a copy of this software and associated documentation
files (the "Software"), to deal in the Software without restriction,
including without limitation the rights to use, copy, modify,
merge, publish, distribute, and/or sell copies of the Software,
and to permit persons to whom the Software is furnished to do so,
provided that the above copyright notice(s) and this permission
notice appear in all copies of the Software and that both the
above copyright notice(s) and this permission notice appear in
supporting documentation.

BCDL is however distributed without source code, which makes modifying it at least VERY VERY difficult, and most probably there are not so many programmers with adequate knowledge to even attempt doing it.

Original page of Reanimatolog:
http://bootcd.narod.ru/
seems to be unreachable, I will try to contact Wolfgang Brinkmann, who hosts the German/English mirror:
http://www.wolfgang-brinkmann.de/bcdw/index_e.htm
and ask him if he has any means to let Reanimatolog see this post.

Finally my requests (in order of preference):
1) If anyone has a way to let Reanimatolog read this (and hoping that he will like the idea), please do so.
2) If anyone has the time and knowledge to understand what and how BCDL works and write a similar loader for USB DOS drivers, please post here, I will try (I am not at all a programmer) to my best knowledge to help him, at least in testing it and report
3) If anyone has a better idea on how a similar BIOS extender for USB support can be added to Grub4dos, please post it here


jaclaz

[Ed_P]

Since a few days have passed since this post of mine:
http://www.911cd.net/forums//index.php?s=&...st&p=121664
with no related reply, but it is well possible that it was not seen by any people as it is in the huge Dietmar's USB thread,

The disadvantage of "keeping everything together".

I already found, in my experiments with Grub4dos, that SBM (Smart Boot Manager) code does not boot ALL tested CD-Roms, whilst using the BCDL (I used version 2.01, but the basic idea should be the same) even those CD images were booted succesfully.

How are you overcoming the shortcoming that I've experienced that BCDL only works if run from a floppy? And where did you find "version 2.01"? Are you using the XCDROM.SYS driver with BCLD? (http://www.ultimatebootcd.com/forums/viewtopic.php?p=1804)

Original page of Reanimatolog:
http://bootcd.narod.ru/
seems to be unreachable,

FYI I was just there. It could be a timing or bandwidth issue that you hit.

[jaclaz]

The 2.01a version is inside latest version of BCDW:
http://www.bcdwb.de/bcdw/index_e.htm
http://www.bcdwb.de/downloads/bcdw201a.rar

The original russian pages do work now , but it seems like I cannot find a e-mail address on them.

How are you overcoming the shortcoming that I've experienced that BCDL only works if run from a floppy?

I boot a floppy image through Grub4DOS/Memdisk

Are you using the XCDROM.SYS driver with BCLD?

No, did not try the idea suggested on that thread (IMHO more questions than answers in there, like in this thread )

jaclaz

[silverni]

Sorry for my English, automatic translation.
The BIOS extension for USB already exists, they are the drivers DOS USB, a very powerful extension; the problem is that GrubDOS sees alone device BIOS, it doesn't recognize device managed through ASPI manager. The optimal solution would be:
1. Boot DOS from A/C/CD/USB1
2. DOS loads USB drivers, which mount a device on USB2
3. GrubDos starts NTLDR/Setupldr.bin on device USB2 (currently not possible, GrubDos doesn't see device managed through the drivers USB DOS).

Question for Tinybit: it is possible to extend GrubDos to see device managed by driver DOS USB (Panasonic-MottoHairu, Duse, etc.)

------------------------------------------------------
Scusate il mio inglese, traduzione automatica.
L'estensione BIOS per USB esiste già, sono i driver DOS USB, una estensione molto potente; il problema è che GrubDOS vede solo device BIOS, non riconosce device gestiti tramite ASPI manager. La soluzione ottimale sarebbe:
1. Boot DOS da A/C/CD/USB1
2. Il DOS carica i driver USB, i quali montano una memoria su connessione USB2
3. GrubDos avvia NTLDR/Setupldr.bin su unità USB2 (attualmente non possibile, GrubDos non vede unità gestite tramite i driver USB DOS).

Domanda per Tinybit: è possibile estendere GrubDos in modo da vedere device gestiti da driver DOS USB (Panasonic-MottoHairu, Duse, etc.)

[jaclaz]

@silverni
Yes, this is EXACTLY what BCDL does (loading a DOS driver), the "clever" part is that it does it directly, without loading DOS.

Update:
Mr. Brinkmann was kind enough to reply to my e-mail.
As I suspected, Reanimatolog is nowhere to be found since a couple of years...

Since the BCDL license expressly allow to "modify" and redistribute files, next step should be try and understand how BCDL works and modify it to invoke the various DOS USB drivers.

By combining the help from BCDL 1.50z and a quick "peek" inside the code of both BCDL 1.50 and 2.01, it appears that there is already a provision to load some USB drivers:
from the 1.50z help:

TRIES to load the following drivers (if present on the floppy):
vide-cdd.sys (1)
usb_cd.sys (3)
usbaspi.sys + aspicd.sys (3)
aspi8u2.sys + aspicd.sys (4)
aspi8dos.sys + aspicd.sys (2)
aspi7dos.sys + aspicd.sys (4)
aspi4dos.sys + aspicd.sys (3)
aspi2dos.sys + aspicd.sys (3)

(1) - works almost always
(2) - works sometimes
(3) - did not work for the author, but may work for you
(4) - no information

Strings in BCDL 1.50z:

VIDE-CDDSYS /D:NOT_USED
USB_CD SYS /D:NOT_USED
USBASPI SYS /r
ASPI8U2 SYS
ASPI8DOSSYS
ASPI7DOSSYS
ASPI4DOSSYS
ASPI2DOSSYS
ASPICD SYS /D:NOT_USED

Strings in BCDL 2.01a:

BCDL-IDESYS /D:NOT_USED
VIDE-CDDSYS /D:NOT_USED
GSCDROM SYS /D:NOT_USED
ATAPICDDSYS /D:NOT_USED
ASPI8U2 SYS
ASPI8DOSSYS
ASPI7DOSSYS
ASPIEHCISYS /int /all
ASPIOHCISYS /int /all
ASPIOHCISYS /int /all /D1
ASPIOHCISYS /int /all /D2
ASPIOHCISYS /int /all /D3
ASPIUHCISYS /int /all
ASPIUHCISYS /int /all /D1
ASPIUHCISYS /int /all /D2
ASPIUHCISYS /int /all /D3
ASPI1394SYS /int /all
DUSE EXE /D:NOT_USED
ASPI4DOSSYS
ASPI2DOSSYS
USB_CD SYS /D:NOT_USED
USBASPI SYS /r
ASPICD SYS /D:NOT_USED

jaclaz

[FM_81]

@jaclaz
Your idea is so nice, that I must test this! But first of all, I'd no success, the problem comes from a little other side.

1. Took a BCDL-floppy (2.01a), and added all the ASPIOHCI.SYS, ASPUHCI.SYS and all the others, whatever the names ...
2. Booting from there, and (a real bootable) USB-HD was recognized, on 2 different machines, with two different drivers, without loading any other DOS-stuff.
3. But than the process stops with "Can't boot from CD, press ..."

I think I know why, but I didn't know, what do about: BCDL is made for CD-drives, and if I'd tested this with an USB-CD (but I didn't have such one), may be it had worked? But the bootprocess of CD is quite different (pointer @ sector 16 etc.) from those of HD's with their real MBR.

So I'm not really gone a step, but if these info helps someone .... ?

[jaclaz]

@FM_81

Yes, it appears that BCDL invokes a USB Driver+BCDL-IDE.SYS

As said BCDL-IDE.SYS appears to be a "custom made" driver, but the vide-cdd.sys in version 1.50z is actually a "normal" generic CD driver that was popular in 1995, so, at least in 1.50z, but I cannot see why it should have changed in 2.01, what BCDL does is exactly what two lines in config.sys or in autoexec.bat "normally" do.

Check the USB dos driver disk you can get here:
http://johnson.tmfc.net/dos/usbdrv.html
(it has all of those drivers loaded in autoexec.bat via DEVLOAD)
or this page here:
http://www.bootdisk.com/usb.htm
(that has the same drivers in config.sys)

Now, there are four possible ways, as I see it, mind you this is more "common sense" that actual knowledge of what happens :
1) Delete BCDL-IDE.SYS and see what happens
2) Replace BCDL-IDE.SYS with a "dummy" file and see what happens (mostly like the empty sfc.dll that helped solve the problem of WFP in Fred Vorck's tutorials, HFSLIP and NLITE)
3) Replace BCDL-IDE.SYS with a FAT (16 or 32) or NTFS filesystem driver for DOS (if they exist)
4) Understand the way the first (the USB one) driver is loaded and prevent the loading of the second one.

jaclaz

[FM_81]

I think, this "CD specific "is "hardcoded" in the BIN-file, I'd already tested this (all drivers deleted, excluding the one that works).
Also I changed the text in the BIN-file, that only the one string remains, that file really exists. Also no success, HD was recognized, the stop with same message.
That means the BCDL.BIN, not the specific driver, is the key of all!?
But never seen this source-code, on the other side, I'm not a programmer :-(

[jaclaz]

On second thought, my point #3 in my previous post is utterly wrong, the "filesystem driver" for CD should be MSCDEX.EXE, which is not at all involved in the BCDL process, so the "something" that should substitute BCDl-IDE.SYS should probably be something that loads raw sector 0 (MBR if "HD" or bootsector if "FD") of the USB Mass Storage device.

jaclaz

[FM_81]

But what about an USB-CD, has ever anyone tested this, and did this work?
If not: we're completly wrong,
if yes: there is a reason, to go on this way!

[jaclaz]

Yes, you are perfectly right.

I have spent last "free" hours of mine researching on DOS USB drivers and the way they are used in BCDL, and I am near to post a tentative version.

At the moment I only succeeded in finding the way to write in the initial part of BCDL.BIN a different name and switches for the invoked driver, NOT much, but it's a start.

The first part of the file is a "list" of files to be called, where the name of the file is written in a "fixed" 8.3 format, padded with spaces (20hex), this is why this one, which name has already 8 letters appears as:
BCDL_IDESYS
and this one, that has 4 letters appears as:
DUSE[Space][Space][Space][Space]EXE
Each entry is separated by a three bytes sequence (0D0A00).

I have understood the method Reanimatolog used, is a kind of the "fallback" in Grub/Grub4dos menu.lst:
BCDL (2.01a) tries:
1) Loading BCDL-IDE.SYS, IF failed:
2) Loading VIDE-CDD.SYS, IF failed:
3) Loading GSCDROM.SYS, IF failed:
4) Loading ATAPICDD.SYS, IF failed:
5) Loading the various ASPI*.SYS, with various parameters, IF failed:
6) Loading DUSE.EXE, IF failed:
7) Loading ASPI2DOS.SYS, ASPI4DOS.SYS and USB_CD.SYS, IF failed:
8) Loading USBASPI.SYS and ASPICD.SYS, IF failed
9) Prints the Error message Cannot boot from CD. Press any key to boot from HDD or ESCape to reboot

I already succeeded in rewriting the VIDE-CDD.SYS as 1st entry, and it works flawlessly with a normal IDE CD.

Now, the ones in point 8) are the "famous" Panasonic Motto Hairu drivers, that are usually very effective.

I am researching about their documentation, but it appears that they are composed of two parts:
1) USBASPI.SYS which is the ASPI extender and supports various command-line parameters, in BCDL it is used /r that "should" be a stay "Resident" switch, but there is aso the /v (Verbose, should print out more info when loading) and /w (Wait, should give more time for connection to the USB device) and a few others that I a investigating.
2) The actual device driver that can be EITHER or BOTH of:
2.1) ASPICD.SYS, which is the device driver for CD's
2.2) DI1000DD.SYS, which is the device driver for HD's or also the Novac "Ninja" driver

Now, if I can start the driver in point 2. it should not be impossible to find inside the BCDL binary the code that invokes the CD bootsector and change it to invoke the MBR of a stick/hd.

The beta 2.01a appears to be, well, a BETA, the syntax for the DUSE.EXE line appears to be wrong, but this of course can be easily fixed.

I plan next week to "release" a version bundled with a small patching program that can help experiment with various drivers, one by one, but as FM_81 pointed out we need someone willing to experiment with a USB-CD.

As I see it, even if we do not succeed in the "disk" booting, even if we succeed in "cd" booting would be a nice add-on to the community.

Any taker?

jaclaz

[dog]

I have a couple of 5 1/4" usb enclosures and machines of varying vintage - happy to test stuff if this helps.

[nomadh]

I'll have to test this stuff out. I've only ran usb-cd drivers from floppy or int-cd before. Had general success in fact its my fallback position if I can't boot from usb-stick. I found the DUSE drivers were not able to boot any of my different PCs using either ver. In fact it usually caused a full lockup on boot. The iomega boot disk image floating around the internet booted every pc I tried. Perhaps ranging back to pentium2 days. I suspect it may work on 486 even but haven't tested but I may have one still in a junk box.
I did run into one maddening issue booting th
I'll hook up an external cd and do some testing.

[jaclaz]

Small update.

I am still far from anything even remotely working, but took a lot of time finding the various drivers (and various versions) scattered around the net and have now what I think is a fairly complete collection of it, together with many posts that have helped me clear my ideas on the usage.

For the record, the bootdisk here:
http://johnson.tmfc.net/dos/usbdrv.html

has most of them, but there is an error in the autoexec.bat regarding DUSE drivers, pay attention to it if you are testing it, the DUSE drivers need their own driver loader instead of the "generic" DEVLOAD.

After much thinking about how to do this I have come to this general idea (this is still limited to the "first" goal i.e. making a "universal boot from USB CD-ROM floppy):
1) Floppy with Grub4dos as bootloader
2) Choice between loading 3) or 4)
3) "Normal" DOS with a choice (menu) system to try all possible drivers combinations and parameters
4) "Alternate" DOS to be accessed once a working combination has been found, capable of:
4a ) "Injecting" (patching) the chosen driver inside BCDL residing on a 160 Kb floppy image in RAM or alternatively, from a gzipped 1.44 Mb floppy image)
4b ) Booting from the floppy image in RAM which should (hopefully) boot the CD from BCDL
5 ) Possibility to save back the modified image to the floppy

Using the RAM floppy should help in minimizing writes to the "real" floppy and possibly speed up everything.

What do you think of this approach?

Also anyone has a suggestion for an EASY, FREEWARE small app for the menues?

I have found several nice old DOS ones, and still testing them...

jaclaz

[Former_Lilas_post]

@jaclaz

For the menu, is wbat.com as used in many DOS floppy boot images a candidate ?

[Former_Lilas_post]

@jaclaz

I tried to understand the purpose/utility of the function you talked about in this thread. Please rectify or clarify what I understand below.
I understand that bcdl.ima (floppy disk image) allows to load various existing DOS drivers without loading DOS, so on USB key boot, with syslinux/grub
installed on the USB key, we can indirectly boot CD/DVD ? If this is correct, the function is only useful and needed for PC/BIOS that cannot boot
directly from CD/DVD, isn't it ? This probably why you mentioned Smart boot manager that can be booted from diskette than it offers to boot from CD-ROM
(indirect).

Other question is : since GRUB4DOS can also have in menu.lst the option to boot from CD (chainloader (cd0) syntax I believe), so if the USB key has GRUB4DOS
installed, why do we still need the bcdl function ? Maybe I missed something for full understanding...

[jaclaz]

Yes,
the idea is to (hopefully) find a way to solve the problem for people that have a PC which BIOS cannot boot from USB.

If it will eventually work, the idea is to boot grub4dos from an alternate media, a "kicker" floppy, the HD or a non-USB CD-ROM and pass control to a floppy .ima containing the modified BCDL and the drivers.

As is, BCDL invokes BCDL-IDE.SYS and boots through it the standard ATA/ATAPI CD-ROM.

First step will be to make it load USB drivers and boot through them a USB CD-ROM.

Second step, if first one works, would be modifying the code to use the same USB drivers to boot a HD-like USB device instead of CD-ROM.

If the whole thing works, the effect would be the ability, having somehow booted from Grub4dos on the HD, the ability to boot WITHOUT changing BIOS boot order settings, from ANY other device, even on machines that already support USB booting. (Right now this can be done for "normal" floppies, and ATA/ATAPI CD-ROMs but not for USB devices)

jaclaz

[mdp784]

I've been experimenting with this idea for a couple of months now. I wrote the "ASPI BIOS", which, like BCDL loads DOS drivers directly. It's restricted to ASPI drivers such as Panasonic's USBASPI.SYS as these provide more control and are not file system specific.

So far I've been able to boot a 7 year old PC (USB 1.1) and a 5 year old laptop (USB 1.1) from a UFD (BartPE, Knoppix, Memtest+ 1.70, Win98 DOS) and a USB CD-ROM using Iomega's ASPIUHCI.SYS and ASPIOHCI.SYS respectively.

On newer machines (where only the Panasonic driver will work) I have not been successful. Win98 DOS and memtest boot fine but BartPE and Knoppix both hang during booting. One of the test machines supports USB booting in the System BIOS and both BartPE and Knoppix boot successfully through BIOS support (so the problem is not related to the machine).

The other big problem with this approach is memory. BCDL loads the drivers at the top of conventional (base) memory. While this is fine for small drivers like BCDL-IDE.SYS (which no doubt has been written to be as small as possible) it is no good for larger drivers (like those for USB support). The problem is that some boot code assumes that all the memory is available and will therefore overwrite the driver code, causing the system to hang the next time an attempt is made to execute it.

I'm now looking into the possibility of writing a small USB driver for this.

I have no plans to release the code I have written so far, it's not in a useful state yet.

Any thoughts?

[jaclaz]

Well, as I see it, you have already done the BIG part of the work.

Before this, someone without BIOS support for booting from USB would have had NO chance whatsoever.

Being able to boot to a DOS-like OS is ALREADY a great step, the booting of BartPE/NT based system should be resolved by proper "specific" workarounds, like RAM booting or using the methods by Dietmar, euhenio and aec.

I know that having "everything and now" is better, but having "something" now is much better than having nothing.

If I were you I would try to get in contact with Tinybit, I am sure he will be able to modify the grub4dos code as to allow for the "peculiar" memory usage you detailed, or maybe found a different workaround, as long as you are willing to release your code as Open Source/GPL.

If you need help in this, I may act as "communication officer".

However, if you would release (independently from the above) a test version of your code, most probably members here could help in widening the range of test machines and support you with bug reports.

jaclaz

[mdp784]

@jaclaz

If I were you I would try to get in contact with Tinybit, I am sure he will be able to modify the grub4dos code as to allow for the "peculiar" memory usage you detailed, or maybe found a different workaround, as long as you are willing to release your code as Open Source/GPL.

Unfortunately it won't be that easy.

The whole thing works by providing an Interrupt 13h (BIOS Disk Services) handler which generates SCSI commands and sends them to the device through the ASPI driver. The boot code invokes the handler to access all BIOS supported disk devices. This means that the handler and the driver must be available (and working) up until the time that the loaded OS takes over control of the devices.

The ATAPI CD-ROM support in grub4dos works in a similar way to BCDL (loads at the top of conventional memory and like BCDL only requires a small amount of memory (source code states 6KB) therefore it works fine.

USB driver sizes are much larger:

ASPIUHCI.SYS - 50606 bytes
ASPIOHCI.SYS - 49242 bytes
ASPIEHCI.SYS - 49772 bytes
USBASPI.SYS - 39179 bytes

Drivers this size cannot be loaded at the top of conventional memory becuase of the risk of an overwrite. I found a way around this by using the excellent UMBPCI.SYS. This driver enables write access to the unoccupied blocks of upper memory (UMBs). The driver and Int13 handler can then be loaded there, out of the way. It works very well. The problem is that the amount of contiguous upper memory available varies from machine to machine depending on add-on cards (SCSI etc.). I have access to some machines which have 100KB+ available and another that has only 36KB. From above you can see that 36KB is not enough for any of the drivers to load at all, hence the next step - try to write a small USB driver that can load.

Ultimately, I think I will have to change the design of the software, so that it requires less memory. Who knows, maybe loading at the top of conventional memory will become feasible again.

mdp784