Page 1 of 1

GTOS

Posted: 23 Dec 2019, 12:04
by marcelk
[This is a good moment to split the expander board software development from the hardware thread.]

To make the SD card reading somewhat meaningful, we must evolve CardTest into a tiny "card operating system". It can be loosely modelled after CP/M and MS-DOS. But an OS is a big thing. The challenge is to keep this minimalistic, but extendible. While at first there's no need for FAT32 writing it's good to think a bit about the organisation and the API. But let's not go overboard with features. Otherwise it will be much better to implement v8080, ADM3A emulation and simply run CP/M itself on the Gigatron and be done. But that's a too big project for me now (2-3 months).

I figure the following programs can be forked from CardTest and make up a tiny system. Let's call the ensemble GTOS (GigaTron OS...):

CardBoot
The ROM-embedded boot loader called from Reset. It will try to load System.gt1. CardBoot isn't a full blown FAT32 implementation, nor an efficient GT1 loader. Just the bare minimum. If System.gt1 can stay under 4K, CardBoot can even ignore the allocation tables.

System.gt1
The GTOS' core. A kind of resident RAM library for Basic Operating System Services ("BOSS" is a cooler name than "BIOS" :-)). Initially just some video terminal I/O based on the ubiquitous PrintChar and Newline functions, plus FAT32 directory and file reading. No card writing at first. Additionally it can load a second program called Command.gt1. It automatically does this after startup.

GTOS defines an API that should be stable over time. It must also have a minimal footprint to leave most memory available for user programs. If the system grows, GTOS could take ownership of the banked memory above 64K. It can then maintain a file system cache there. It can also hide parts of itself in there so that most of the "live" 64K address space remains available for user programs.

I suggest to allocate $200..$4f9 to the visible part of GTOS, and $500..$6ff for a sector buffer and other single-use purposes. Zero-page use must be minimised.

Command.gt1
A tiny command line shell that builds on top of the services provided by GTOS. It can search for a file whose name is typed by the user. When it detects the file is a GTOS-compatible command it will load and run it directly. GTOS commands can be recognised by the load address of their first segment (in my draft notes that would be $503). GTOS commands use GTOS services to do something interesting. Then they return to Command.gt1 with the vCPU RET instruction, or through the GTOS entry point that reloads Command.gt1. This makes it possible to extend GTOS with new commands in the same way as CP/M and MS-DOS.

If the file isn't a GTOS command, Command.gt1 should reset the video terminal, load the GT1 in a similar way as Loader and launch it. That effectively ends the GTOS session. This launch functionality can also be delegated to a separate Launch.gt1 application: its overall function is really different from loading commands, because general GT1 files will load almost anywhere. And they typically expect an unmangled video table. Commands behave the opposite: there you want to preserve the updated video display between typed commands. (Remember that text scrolling is done my changing the video display list instead of moving text data.)

The Dir command can be a built-in. That's not much extra with respect to what Command.gt1 must already be capable of. At least, for as long as Dir only displays names, not attributes.

List.gt1
This would be a GTOS command that lists the directory, showing its date attribute and file size. All the functions are already in CardTest.gcl, so we can evolve it from there.


This may sounds like a big project, but all building blocks for the above are already present in CardTest and other programs. We just have to reorganise them.

Re: GTOS

Posted: 22 Jan 2020, 16:26
by xopr
I'm trying to catch up on all the awesome new features and dug up a perfboard SD card interface that I haven't touched for 11 years, back when I didn't even knew what Arduino was (I was using MyAvr with Petit FatFs)

So when I tried it on the Gigatron (without any sign of life), I decided to test with Arduino's default SD library, and I had some interesting discoveries along the way. (The short conclusion is that I'd swapped GND and SS :oops: )
  • mkfs.fat auto-selects the best filesystem type for its size (fat 12,16 or 32); force with -F32
  • force formatting a 16MB MultiMedia Card as FAT32 is impossible (it needs to be at least 33MiB)
  • Arduino's SD library doesn't support MMC; it doesn't call CMD1
  • Only my 8GB SDHC card (type 2) seemed to work (I had both MMC and SD type 1 cards that failed on MBR and Volume ID)
For the MMC, MBR errors might be due to an initialization error (they need CMD1 and I saw some remarks about it in CardTest), but for the SDC, it first gave a Vol.ID FFFF FAILED, and after some partitioning and formatting it errored out with Vol.ID 0000 FAILED; Maybe it is related to the alignment of the partition?
The one that worked had the LBA flag on and its first sector was at 2048. I'm testing with a 128K system.

So if I would have to ask one question, it would be: what are the current type and format of cards that people were able to use successfully?
I know it might be a really small set of people/cards, but any information might be useful to demystify this subject :).
I know it's not really GTOS specific (sorry for hijacking; I didn't know a better place to mention it), but since it is initialized by the OS, it might be convenient to mention the edge cases.

I successfully was able to run a couple of small programs like HelloWorld, Blinky and Terminal by renaming them to system.gt1 on the SD card, which is a really cool experience!

Maybe not related to GTOS but to verify that all of the SD cards worked in my makeshift reader, I had to add MMC support on the Arduino SD library (with some help); I modified the following files (in the ~/Arduino/libraries/SD/scr/utility/ folder):

In Sd2Card.cpp, changed the complete while loop:

Code: Select all

  while ((status_ = cardAcmd(ACMD41, arg)) != R1_READY_STATE) {
  ...
  }
into:

Code: Select all

  status_ = cardAcmd(ACMD41, arg);
  while (status_ != R1_READY_STATE) {    // check for timeout
    // check for timeout
    unsigned int d = millis() - t0;
    if (d > SD_INIT_TIMEOUT) {
      error(SD_CARD_ERROR_ACMD41);
      goto fail;
    }
    // Switch to CMD1 if the card fails to recognize ACMD41
    if (status_ & R1_ILLEGAL_COMMAND)
        type(SD_CARD_TYPE_MMC);
    status_ = (type() != SD_CARD_TYPE_MMC) ? cardAcmd(ACMD41, arg) : cardCommand(CMD1, 0);
  }
In Sd2Card.h, added:

Code: Select all

/** Old MMC card */
uint8_t const SD_CARD_TYPE_MMC = 4;
In SdInfo.h, added:

Code: Select all

/** SEND_OP_COND - Initiate initialization process. */
uint8_t const CMD1 = 0x01;

Re: GTOS

Posted: 22 Jan 2020, 18:02
by marcelk
There is an interesting thing going on with the sequence for entering SPI mode on these devices: for some SD cards CMD0 seems to send its reply without driving the bus in both directions: as if those cards still only have a weak pull-up (or none) on the MISO line during that response. Our pull-down resistor can therefore corrupt the CMD0 response.

It's not just that we see more (or only) zeroes coming back. It isn't that simple. One card replies with 0x3F instead of 0x01, so fewer zeroes. That is very weird if you think about it. But this card responds ok with 0x01 when there is a pull-up on MISO. I suspect that this specific card monitors the MISO line, and changes its behaviour based on what it detects. So this line is not purely an output for that card. I have no other explanation.

The good news is, for all commands after CMD0, all SD cards I tested drive MISO properly in both directions up and down. They really do enter SPI mode with the pull-downs. This is also what you expect them to do. I don't see this in the MMC descriptions however (and I don't have an MMC adapter to test them).

Therefore my approach will be two-fold:
  • Ignore MMC cards
  • Ignore the CMD0 reply from SD cards. Simply continue with the next commands and check those responses
This isn't in System.gt1 and in CardBoot yet. The reason is that I got stuck in choosing a system call convention. It turns out you can't do that in a manner that is an analogue of the CP/M and 86-DOS convention (being: one register for the function number, a second register for the function argument). In vCPU, functions with multiple arguments don't lead to elegant call sequences, and I really want an elegant call sequence... I also want to avoid having a big GTOS jump table in zero-page: there should be just a single entrance point to GTOS. Over the past weeks I have grown an idea on how to resolve that issue, but now I need to find time to pick it up again. Sometimes life gets in the way of important retro computing projects.

Re: GTOS

Posted: 22 Jan 2020, 21:25
by marcelk
xopr wrote: 22 Jan 2020, 16:26 for the SDC, it first gave a Vol.ID FFFF FAILED, and after some partitioning and formatting it errored out with Vol.ID 0000 FAILED; Maybe it is related to the alignment of the partition?
The one that worked had the LBA flag on and its first sector was at 2048. I'm testing with a 128K system.
It might be that there's still an addressing bug in the FAT32 calculations. I have one card with a similar Vol.ID error. It probably reads the wrong sector. I still have to plough through that card's hex dump :-)

[Edit: created GitHub issue: https://github.com/kervinck/gigatron-rom/issues/161]
So if I would have to ask one question, it would be: what are the current type and format of cards that people were able to use successfully?
I know it might be a really small set of people/cards, but any information might be useful to demystify this subject :).
I know it's not really GTOS specific (sorry for hijacking; I didn't know a better place to mention it), but since it is initialized by the OS, it might be convenient to mention the edge cases.
I now plan only for SD cards, only FAT32 formatted, only the first partition, only 2 FAT areas (that is standard), only 512-byte sectors (that is standard), and only LBA. But it seems we can accept both 0x0B and 0x0C filesystem types, even though 0x0B is advertised as non-LBA. In reality they appear as LBA anyway (it doesn't make any sense otherwise).

I successfully was able to run a couple of small programs like HelloWorld, Blinky and Terminal by renaming them to system.gt1 on the SD card, which is a really cool experience!
That is incredibly cool to hear! And that boot loader is still horribly inefficient, processing the file byte-by-byte instead of loading segments in one go.

Re: GTOS

Posted: 10 Feb 2020, 22:38
by marcelk
From skimming over the older CardTest code, I conclude that byte addressing isn't detected correctly. And even it if were, it wouldn't work anyway, because the way it is handled further on isn't correct. I believe this is consistent with the observations so far, including jwolfram's today. This is purely a software issue to be solved in GTOS at some point perhaps. It isn't related to the expander circuit itself.

The first fix is to reject byte addressing cards, instead of failing later on.