Saturday, August 5, 2017

FreeDOS for OEM

Jerome Shidel contributed this article for the FreeDOS summer coding blog challenge, about how to use FreeDOS for OEM PCs.
Different computer manufacturers will sometimes include FreeDOS as a pre-installed operating system option. This is always great to see! And FreeDOS, like any DOS, is pretty easy to set up as a pre-installed operating system.

When creating the FreeDOS 1.2 installer, I thought it would be good to keep this subject in the back of my mind. How can we make FreeDOS easier for original equipment manufacturers (OEMs) to include as a pre-installed operating system?

All of those computers, the ones that ship with their only installed operating system being FreeDOS, they should be considered. Yes, I do realize that mostly they are sold outside of the United States as a low cost option. And, users will most likely just be installing a copy of Windows or Linux onto them. But, I figured there may be some people that get them to run FreeDOS. So, I kept that in mind while designing how the installer handles certain things. Nearly always, I attempted to maintain an OEM PC compatible and extensible design. I knew it should work. But, I was very busy prior to the 1.2 release and never got around to testing it.

Now that FreeDOS 1.2 has been out for a while, I wanted to come back to the subject. I finally have tested it, and wow—it is awesome!

So, here is the quick way I made an OEM PC style setup with FreeDOS 1.2:
  1. Create a USB fob drive using the USB "Full" installer (FD12FULL.img).
  2. Boot that USB fob drive on the PC.
  3. When the installer launches, select the language then exit to DOS.
  4. Run FDISK 2
  5. Create your FreeDOS Recovery Partition. In my test, I used 1024MB.
  6. Exit FDISK and reboot the computer.
  7. When the installer launches again, select the language then return to DOS.
  8. Run FORMAT D: /q/u/v:FD-RECOVERY
  9. Run XCOPY /E C:\*.* D:
  10. Run SYS D:
  11. Run FDISK 2
  12. Activate the Recovery Partition on the D: drive.
  13. Create the user's big main FreeDOS partition, but do not activate it.
  14. Shutdown, then clone the drive a couple million times.
Once done, it has a couple really cool effects:
  • The first time the PC is booted, the user gets to install FreeDOS. They even get to pick their language and keyboard settings for their install. Afterwards, when the system reboots, it boots into their install.
  • The C: drive is that big user partition and the D: drive is the Recovery Partition. The Recovery Partition has all of the packages from the FreeDOS 1.2 distribution. So, you can use FDIMPLES to install and remove more packages with out needing to insert any additional media.
  • OEM vendors could add additional packages. Vendors might do this for networking and sound drivers. Those custom packages could be configured to be automatically installed.
There was only one issue of note that I found. If the user wishes to reinstall over their existing installation, using the Recovery Partition, they will need to change their active partition using FDISK to the Recovery Partition. Otherwise, the SYS transfer will fail. This actually has to do with force-updating of the MBR code, and not the simple SYS transfer. You can also avoid this problem by running the FreeDOS Installer in Advanced mode and “Not transferring the system files.” But, it is easier just to re-activate the Recovery Partition and reboot.​

Thursday, July 20, 2017

The FAST and SOFA compilers

As part of the FreeDOS Summer Coding Blog Challenge, Bruce Axtens writes on his blog about the history of the FAST and SOFA compilers for FreeDOS.

If you don't know about FAST and SOFA, FAST is loosely based on Basic, Pascal, C and Assembler. The programs produced by FAST are very small and very fast. SOFA is an 8088 (and semi-80386) assembler, with syntax based somewhat on A86 and CHASM.

Both FAST and SOFA were created by Peter Campbell, who passed away in 2007. FAST and SOFA have since been released under the GNU GPL.

Thursday, July 6, 2017

How to compile bwBASIC from source

Dr Owain Kenway recently wrote about a problem he discovered with Bywater BASIC, one of the BASIC interpreters we include in the FreeDOS 1.2 distribution. Looks like we made a mistake on our end by including the Win32 version, but Owain shows how to recompile a native DOS version. You can read his how-to on his blog:

Fixing bwBASIC on FreeDOS »

While this isn't necessarily part of the FreeDOS Summer Coding Blog Challenge, I'd like to include it as such because it shows the power of open source software: you can always compile it yourself.

(We'll also update the bwBASIC package on our end. Thanks, Owain!)

Wednesday, July 5, 2017

How to support different spoken languages in FreeDOS programs

All summer, we are doing the FreeDOS Summer Coding Blog Challenge, where we ask you to write and share some tips that new developers can use to get started in FreeDOS. I thought I'd start by writing about the Cats and Kitten libraries to support different spoken languages in your programs.
When you write a new program, you probably don't think about spoken languages other than your own. I am a native English speaker, so when I write a new program, all of my error messages and outputted text is in English. And that works well for the many people who have English as their native language, or who know enough English as a second language to get by. But what about others who don't speak English, or who only know a little English? They can't understand what my programs are saying.

So in the early 2000s, I decided FreeDOS should have a system to support different spoken languages. This type of multi-language support is called Internationalization (or "I18N," because there are eighteen letters between the "I" and "N").

I looked around at how other popular systems had implemented multiple spoken languages. The standard Unix method is with a set of C library functions built around language "catalogs." A catalog is just a file that contains all the error messages and other printed text from a program. In the Unix method, you have a different catalog for every language: English, German, Italian, Spanish, French, and so on.

My first FreeDOS I18N library was a stripped-down implementation of the Unix method. This is a very simple method. Every time you want to print some text in the user's preferred language, you first look up the message string from the catalog using the catgets() function—so named because it will get a string from a message catalog.

In Unix, you use catgets() this way:

  string = catgets(cat, set, num, "Hello world");

This fetches message string number num from message set set, from language catalog cat. The organization of messages into sets allows developers to group status messages into one set (say, set 1), error messages into another set (such as set 7), and so on.

Before calling catgets(), you need to open the appropriate language catalog with a previous call to catopen(). Typically, you have one catalog per language, so you have a different language file for English, another for Spanish, etc. Before your program exits, you close any message catalogs with calls to catclose().

If the string doesn't exist in the message catalog, catgets() returns a default string that you passed to it; in this case, the default string was "Hello world."

I implemented a simplified version of these functions in a FreeDOS Internationalization library called Cats. To save on memory, Cats supported only one open catalog at a time. If you tried to open a second message catalog, the call to catopen() would return an error (-1).

Message catalogs were very simple under Cats. Implemented as plain text files, Cats loaded the entire message catalog into memory at run-time. In this way, you didn't need to recompile the program just to support other languages; you just added another message catalog file for the new language. An English message catalog for a simple program might look like this:

  1.1:Hello world
  7.4:Failure writing to drive A:

The same message catalog in Spanish might look like this:

  1.1:Hola mundo
  7.4:Fallo al escribir en la unidad A:

For example, the string "Failure writing to drive A:" is message number 4 in set 7.

The Cats library was a simple way for developers to add support for different languages in their programs, written in C. And because Cats implemented a Unix standard, it made porting Unix tools to FreeDOS much easier. Once you added the calls to catgets(), all you needed to support other languages was a message catalog that someone translated to a different language. And I kept the Cats message catalogs very simple; they were plain text files.

Cats was a neat innovation, but loading the messages into memory was cumbersome because it used streams. I admit that I just did it the quick and easy way. Fortunately, other FreeDOS developers improved on Cats to optimize the loading of catalogs, reduce memory footprint, and add other enhancements. The new library was noticeably smaller, so we renamed it Kitten, also written in C.

Because of the optimizations, Kitten used a slightly different way to retrieve messages. Since Cats only supported one message catalog at a time anyway, Kitten removed the cat catalog identifier. Once you open a message catalog with kittenopen(), all calls to kittengets() assume that message catalog. You only need to make a single call to kittenclose() before you end the program.

Using Kitten made it much easier to support different spoken languages in FreeDOS programs. Here's a trivial example to put it all together:

  /* test.c */

  #include <stdio.h>
  #include <stdlib.h>
  #include "kitten.h"

  int
  main(void)
  {
    char *s;
 
    kittenopen("test");
 
    s = kittengets(7, 4, "Failure writing to drive A:");
    puts(s);
 
    kittenclose();
    exit(0);
  }

This loads a message catalog "test" into memory, then retrieves message 4 from set 7 into a string pointer s. If message 4 in set 7 isn't found, kittengets() returns the default string "Failure writing to drive A:" into s. The program prints the message to the user, then closes the message catalog before exiting.

Typically, you name the message catalog after the program's name. So the message catalog for the FreeDOS CHOICE program is "choice". Kitten searches for the language file in a few locations on disk, and always appends the value of the %LANG% environment variable, which is typically set to the two-letter language abbreviation: "en" for English or "es" for Spanish. The DOS filename for the English version of the "choice" language catalog is CHOICE.EN, and the Spanish language version is CHOICE.ES.

A limitation to Cats and Kitten is that it can only support single-byte character sets supported by DOS. So Cats and Kitten cannot support Chinese or Japanese, but should do fine with most other languages.
You can find Cats and Kitten at the FreeDOS files archive on ibiblio, under devel/libs/cats/

We made three revisions to Kitten, so the latest version is Kitten revision C, which you can download directly as kitten-c.zip.

FreeDOS contributor Mateusz "Fox" Viste wrote a similar implementation for Pascal programs, called Cubs. You can also find it on ibiblio, under devel/libs/cats/cubs/

FreeDOS developer Eric Auer created a command-line version of Kitten, named Localize, so you can provide internationalization support for DOS Batch (BAT) files. Find it on ibiblio, under devel/libs/cats/localize/

Monday, July 3, 2017

FreeDOS Summer Coding Blog Challenge

FreeDOS just celebrated our 23rd birthday on June 29. That's a long time for any open source software project, so while there wasn't anything special about "23," we decided to mark our project's anniversary with a FreeDOS Blog Challenge. For the blog challenge, we asked you to share your "FreeDOS story" about how you discovered FreeDOS or how you used FreeDOS. During the month of June, we shared these FreeDOS stories as "guest posts" on our blog. Thanks to everyone who sent in a story! You can find links to all the stories at "Happy 23rd birthday to FreeDOS!"

Since the blog challenge was such fun, a community member on Facebook suggested a followup. I thought that was a great idea.

Announcing the FreeDOS Summer Coding Blog Challenge! Throughout the summer, share your tips about programming for FreeDOS. Post them on your own blog, or email them to us and we'll do guest posts for you.

What should I write about?
You might write a "how-to" article about how to manage very large arrays in memory, or how to load very large files. Or maybe you could talk about how to optimize a DOS program's I/O operations, such as how to read chunks of a file into a memory buffer using read() rather than using streams. Or you might want to write an article about how to use the conio functions to control the screen and read from the keyboard. Or maybe you could write something a little more "entry level" like tips to use the FreeDOS Kitten library to support multiple languages in your program.

Your "how-to" articles don't necessarily need to be about coding in a language like C or Assembly. If your preferred programming language is Pascal or BASIC, then write something about how to write programs in those languages for FreeDOS. If it's a programming language that's included in FreeDOS, we'd like to include it in the FreeDOS summer coding challenge.

Don't limit yourself to compiled languages, either. You can do lots of clever things at the FreeDOS command line and in BAT ("batch") files. So you could write an article about how to use a batch menu tool, like Jerome's V8 PowerTools included in FreeDOS 1.2, to create a neat interactive "program" as one smart batch file.

We want to hear from everyone! It's not just about developers, or people who contribute to the FreeDOS Project directly. The FreeDOS summer coding challenge is for anyone.
How do I write an article?
If you don't often write for a blog, then writing a FreeDOS "how-to" article might seem a little daunting. But really, it's easy!

I recommend you write your article as though you were explaining it to a friend. If it helps, write a draft in your email program, so you can convince yourself you're emailing someone about programming in FreeDOS. And if you like, you can actually send that email message to me (jhall@freedos…) and I'll use it as a guest post on the FreeDOS blog.

If you have your own blog or website, post your story on your blog, and email me to let me know where to find your article. If you don't have your own blog, I would be happy to post it for you as a "guest post" here. I'll even do light editing for you and take care of formatting.

Don't worry about making it perfect. I can do light editing for you before I put it up as a guest post. As in the FreeDOS Blog Challenge, I'll edit for spelling, grammar, and style before I post your article on our blog. I try not to make changes to content or to the flow of your article—but if I do, I'll always run the changes by you first.
Send us your story by September 30!

If the FreeDOS summer coding challenge works out well, we might do a similar blog challenge in winter, maybe about your favorite DOS applications.

Guest post: Updating BIOS with FreeDOS

Lars Noodén shared this note about using FreeDOS to update the BIOS on a computer. On the FreeDOS website, we like to say that most people install FreeDOS to play classic games, run legacy software, or do embedded development. And for installing FreeDOS, that probably covers almost everyone. But there are a lot of folks who use FreeDOS to do exactly what Lars describes: update the BIOS.

I use FreeDOS only occasionally—but when I have used FreeDOS, it has been a life saver. The most recent time was when I needed to update the BIOS on an old laptop. I was able to make a FreeDOS image containing the BIOS update and then modify GRUB2's configurations to allow me to boot it. The GRUB2 part, as opposed to GRUB1, was hard. The FreeDOS part was straightforward and easy. Here's how I did it:

This was all done on an ancient laptop using a Long Term Support (LTS) version of Ubuntu GNU/Linux for AMD64 computers, which to my consternation used GRUB2.

First, after downloading the FreeDOS boot floppy and unzipping it, I mounted the resulting disk image read-write using the "loopback" device so that I could edit it. I copied the vendor's ".EXE" style BIOS update there so it would be available while FreeDOS was running. I then removed AUTOEXEC.BAT so that I would get the normal command prompt when booting. Then I unmounted the disk image and it was ready to go, so I copied it to /boot/floppy.img and was all set with that. That was the easy part.

Next was the hard part with GRUB2. While it was fairly easy to figure out how to configure the old GRUB1, the new version is not. After much searching online, I ended up "cargo-culting" it. So if you follow this route, try to find and read current documentation. It turned out that I had to first to get memdisk into the /boot directory. The package syslinux at the time could do that—although now the package grub-imageboot seems to do that job, and installing it puts memdisk directly in /boot for you.

Then GRUB2 had to be configured to offer memdisk and the modified FreeDOS boot floppy as an option. That is done by creating a new file, /etc/grub.d/42_custom, and making it executable. In it, I put the following:
 #!/bin/sh
 cat <<EOT
   menuentry "FreeDOS" {
     linux16 /boot/memdisk
     initrd16 /boot/floppy.img
   }
 EOT
Then once update-grub was run, FreeDOS showed up in the boot menu starting with the next boot.

And with FreeDOS now in the boot menu, it was just to reboot the computer, select FreeDOS, and then run the BIOS update from the A: prompt. After the BIOS update ran to completion and I was back in Ubuntu, I removed the GRUB2 entry.

FreeDOS really saved the day in that way, since there was no other way to conjure the equivalent of a bootable floppy disk.

-Lars Noodén

Friday, June 30, 2017

Guest post: Building the FreeDOS installer

Jerome Shidel created our new FreeDOS Installer, which first appeared in the FreeDOS 1.2 distribution. Jerome shared his FreeDOS story:

My FreeDOS story began many years ago in the pre DOS days. An early version of MS-DOS may have been around. But, it definitely was not a thing yet. It brings back fond memories.

At age 9, laying on the living room floor and hoarding the television. The Sinclair ZX80 powered up and its manual open. Teaching myself to program its 1 kilobyte of RAM with that terrible membrane keyboard. It wasn’t long until my father got me the enormous 16Kb RAM add-on module. That was a lot to fill back in those days. Especially when you bumped the ZX80, it would reset that extra memory module.

Next came that incredible Atari 800XL and the Coleco Adam home computers, before my father got a Laser XT and we moved into the reign of the 8086 CPU and MS-DOS. I spent many wonderful winter days sitting up in the attic with a space heater blowing under the desk, waiting for the keyboard to warm up enough for the computer to boot without errors.

After several years, Windows 3.1x became popular. But, I only had limited use for it at the time. I still spent most of my time wearing out keyboards programming odds and ends in DOS.

In 1995, Microsoft broke my heart with Windows 95. It looked so new and cool. I was so excited to give try it on my almost new $5000 notebook. That would be pricey now, imagine that in 1995 dollars. I went over all the requirements. Everything specification was met or far exceeded what it needed. I was ready to rock Windows 95. Or, so I thought.

Part of the way through installation, the notebook all of a sudden went to just a black screen. The install trashed my video BIOS firmware. According to the manufacturer, I would have to send it in and have some chips replaced. Not the patient sort, through a lot of trial and error, I was able to just re-flash the BIOS and get it working again. But, it would never support Windows 95.

At that point, I started to look around the internet for alternates to Microsoft products. I messed around with Slackware Linux and other DOS systems (like PC-DOS).

Even though I eventually grew to accept what happened in the Windows 95 debacle, I never did truly forgive them. I can really hold a grudge. Not even now.

So, I have been aware of FreeDOS since its early days in the late 1990s. But, I really did not use it much back in those days. FreeDOS was still in its early alpha stages. Plus, there were several other DOS distributions and Linux platforms that I had favored at the time. However, I did install some of those early versions and played around with them a little. It found it interesting, that unlike many of the commercial versions of DOS, FreeDOS was not stagnant. It was slowly progressing. I figured that I needed to keep an eye on that crazy FreeDOS project.

Fast forward nearly 20 years…

Generally speaking, I'm not much into any of the social media stuff on the web. So it was kind of unusual that back around March of 2015, I was wasting time on Facebook.

I was looking around at Facebook pages and groups for some of my interests. OpenSUSE, bash, Delphi, Pascal, assembly, FreeDOS… I though to myself:

“Huh?” “What?” “There is a FreeDOS page on Facebook?” “Wow!” “Neat.”

I was quite surprised to find out that there were still several very active DOS communities around the world. It was pretty weird in a cool way. I figured I should do something nice for them.

So, I perused some of the programs that I wrote back in the early 1990s and decided to make some of the stuff Open Source. Most notably was "Program Manager v7.2" a multi-menu program and game launcher. PGM's most recent update was way back in 1992. Yet, it was well received by the FreeDOS community. I decided to do more.

That led me to doing a complete rewrite of PGM using more modern concepts and techniques. I went a little (ok, a lot) overboard. Theme-able, multi-language, custom fonts, screen savers and etc. Like FreeDOS, The Program Manager Eternity (PGME) was reborn to live forever.

During the development of PGME, it was brought to my attention that Jim Hall was looking to create a brand new installer for FreeDOS. Something that looked more modern, easier to install and was powered by batch files. Something that could use some "simple" command line utilities to install FreeDOS.

Hmm, a set of tiny non-memory resident utilities that can interact with each other to create a text mode UI for batch files. All the logic for the batch program to reside in the batch. Use no memory. Yet provide batch files with enough functionality to build a flexible and simple installer. It sounded interesting. Sure why not.

So, I volunteered to create some GPL tools that could do the job. There were a few naysayers that thought it could not be done or just wouldn’t work. But, most were excited that a new FreeDOS release might be coming soon.

Well as you know, one thing tends to lead to another. As the foremost expert on the usage of V8 Power Tools, a set of batch tools I wrote, I volunteered to create the new installer and work began on FDI, the FreeDOS Installer. There were many enhancements and additions to V8PT during the development of FDI. It was a long and slow process jumping back and forth between them as new needs arose. But, the work progressed.

There was a lot of back and forth with Jim during the development of FDI. Lots of design, workflow and other decisions. Plus, coordinating all of the additional languages supplied by the community, the new installer was quite a lot of work.

After dozens of public beta tests and two Release Candidates, we released FreeDOS 1.2-Final on Christmas Day 2016.

Many thanks to the FreeDOS community for all their help during the development of FreeDOS 1.2. There are a wonderful community with many great people. Without their efforts, this release would not exist.

Then of course there is FDIMPLES. Originally, I created it specifically just to provide detailed package selection for the Advanced Mode of FDI. Its only purpose, modify the package list used during installation. But, as go so many plans, it didn't stay there. It was just to cool for installing and removing packages. I have big plans FDIMPLES.

Nowadays, there are several other areas that keep me busy with the FreeDOS project. But, that would be a tale for another day.

-Jerome Shidel