Program Changes and Data Capacity: A Symbiotic Relationship

Memory space in a computer — regardless of how much you have plugged in — is allocated for certain jobs. Two major sections of memory that concern ZX PRO/FILE are the program area of memory and the variables area. When PRO/FILE runs, all files are stored in the variables area. With 16K attached, the files (also called data) go into an array of characters: D$(11000). Since the total amount of memory is limited to 16K by the size of the RAM pack, most of your memory is used to store the 11000 characters of D$. The remainder is used to hold the actual program and a few other things like the system variables and display file.

When you have a fixed amount of memory-16K for example — and 11K is reserved for data storage, you only have 5K left for every thing else. Roughly, you can figure that the computer needs 1.5K just to operate. This memory is used for calculations, system variables, the TV display, etc. All that’s left for the actual program is 3.5K and ZX PRO/FILE uses just about all of that. The more data capacity you have, the less program lines you can hold. A program full of files is no different than one that is almost empty in this regard. It is capacity (the size of D$) that is important.

If you want to add more than a few short program lines to PRO/FILE, memory problems will likely appear. Symptoms include report-4 error halts while the program is operating. This usually happens when the computer is executing a line that requires a little extra RAM SPACE. Inputting long 28 character search commands or moving the edit cursor are common culprits.

Sometimes the computer gets so full that it won’t let you add a new program line. If you try to enter a new line but every time you press ENTER the line just sits at the bottom of the screen, lack of memory is probably the cause. This can happen when you try to EDIT an existing line too–especially if the line you’re trying to edit is a long one.

You can buy a larger memory pack to expand bot data capacity and program space. You can also sacrifice some of your data size to gain programming room. If your modifications are well written, the cost in terms of data capacity is usually offset by your new capability to access the data you have left with greater versatility.

Books tell you that it is a simple matter to expand an array like D$(11000). Just re-DIM it to D$(12000) to gain 1000 more characters. To make it smaller and thus free up some memory space, go the other way: DIM D§$ (8000) for example.

There’s just one hitch. If you have files typed into the array you lose everything when you redimension it. Since adding files is slow, tedious, and an awful nuisance, you should avoid adding them twice like you’d avoid the plague.

Fortunately there is a way to shorten or lengthen an array and still retain the data held in it. The following procedures detail exactly how to do it. Admitedly, they are somewhat involved, but if you have a program full of data, they beat the heck out of retyping everything back in again.

Use these procedures any time you need them. They can be used on any memory size with a minimum of sacrifice. Most of the time you will lose nothing when you cut back. Only when DS is almost full will you risk losing data. You do, however, have control over just which files get the axe.

If you decide to shrink capacity by 1000 bytes and your SPACE OPEN indicator tells you that there are only 800 slots left, 200 bytes will disappear from your files. Go through your files by operating the program. Delete unimportant lines or complete files. When the SPACE OPEN tells you that there are more slots left than the amount you wish to cut back, it’s safe to proceed.

Numbers given in small type are the results I obtained when a “stock” PRO/FILE was used with files added so that P=1242. The results you obtain will be different.

1. With PRO/FILE in the computer, exit from the Main Menu. Press SHIFT l, STOP, and ENTER.
2. Type PRINT P and ENTER. Write down the result that comes on the screen. (1242)
3. Type PRINT LEN DS and ENTER. Write down this result as well. (11000)
4. Now decide what you want your new length for D$ to be and type: LET P= this new length. (8000)
5. Go back into the program by typing GOTO 17 and ENTER.
6. Add a file consisting of just the asterisk followed by a character unique to the data. (I used an inverse “$”)
7. After you close the file, look it up. Use the “*” and the character you chose above as a search command.
8. When the file is printed on the screen, exit from the program again–this time from the Display Option Menu–using the method shown in Step 1.
9. Type PRINT PEEK 16507+256*PEEK 16508 The result you get is the address in memory where this file starts. Jot down your answer. (29347)
10. Type PRINT PEEK 16400+256*PEEK 16401. This result is the address held in VARS, the system variable which holds the address of the beginning of the variables area of memory. Since D$ is the first variable entered, the address shown is the beginning of D$–not the actual data, but the pointers which define the variable’s type, name, and length. Record this address. (21341)
11. Take your answer in Step 10 and peek its value. Ex: PRINT PEEK 21341. The value you get should be 201 which is translated by Sinclair Basic to Signify the name of the first variable in memory. 201 means DS. (this is another “blind faith” principle)
12. Now take the new length decided upon for DS$–8000 using my example in Step 4–and type RAND (this length). Ex: RAND 8000. This action converts the new length for DS into a 2-byte integer. The result is stored in SEED, the system variable that is set by the command RAND. SEED is located at addresses 16434 and 16435. (credit for this trick goes to the Timex User Group of the Boston Computer Society, June ’83 newsletter)

At this point here is what we know:

• The first byte of D$ (21341)
• The length that D$ is to become (8000)
• The address in memory where this new D$ will end (29347)

The first five bytes after address 21341 are pointers which define the characteristics of D$. You can poke these bytes to change its length. The values you give them come from SEED which you just set by typing RAND. The first pair of bytes must hold the new length of DS plus 3. The next byte holds the number of dimensions, and the last pair of bytes holds the actual length of D$. So now you POKE:

POKE 21342, PEEK 16434+3
POKE 21343, PEEK 16435
POKE 21344,1
POKE 21345, PEEK 16434
POKE 21346, PEEK 16435

Now the computer thinks that D$ is 8000 characters long, but there is still the matter of cleaning up the old bytes of D$. This is done by creating a new array, the length being equal to the left over bytes. This would be the original length of D$ minus the new length. Using my example, the length of the new array should be 11000-8000 or 3000. Thus,

14. Type RAND (the old lengtH minus the new length) Ex; RAND 11000-8000 This action puts the length into SEED so that you can create a new array by poking.

The answer you arrived at in Step 9 will be the first byte of this new array. This address holds the name. If the name for D$ is 201, then 202 must be the name for E$. (more blind faith) Therefore using my numbers as examples, you would POKE:

15. POKE the address you arrived at in Step 9 with 202. Ex: POKE 29347,202
16. Now the five bytes that follow (29348 to 29352) are poked just as in Step 13.
    POKE 29348, PEEK 16434+3
    POKE 29349, PEEK 16435
    POKE 29350, 1
    POKE 29351, PEEK 16434
    POKE 29352, PEEK 16435
17. Cross your fingers because now you are going to find out if you did everything . correctly. You should have a shorter D$ and a new E$. Find out by typing:
    PRINT LEN D$ (8000)
    PRINT LEN E$ (3000)
18. To free up the memory that E$ occupies, make it smaller by typing: DIM E$(1)
19. Reset the variable P to its original value. Type: LET P= your answer in Step 2.
20. Go back into the program and check your files. GOTO 17 and ENTER.

Everything is intact. The SPACE OPEN indicator reveals that there is less file space. You have more room to add extra program lines. It is important to remember that you have an E$ array in memory now. You can utilize it in your modifications. If you ever reduce capacity again, be sure the new array you create is named something different. In Step 15, use 203 to create an F$ instead of another E$. You might crash the system if you have two arrays both with the same name. I’m not sure.

If all this “pointer poking” seems more like a combination adventure game in the memory caverns of your computer and Russian Roulette, hang onto your hats. Here’s how to use the same idea to lengthen an array without losing data. These steps convert your present 16K program into a full blown 64K data base.

1. Load the 16K PRO/FILE into the computer that has had Ramtop poked to reflect a full 64K of RAM.
2. Break into the listing following Step 2 of the previous section.
3. Type PRINT P and record your result. (1311)
4. Type DIM E$(1000,31) and ENTER. This action reserves 31000 bytes for data.
5. Enter RAND (PEEK 16404+256*PEEK 16405)-(PEEK 16400+256*PEEK 16401)-4. This sets SEED to the total number of bytes in the variables section of memory. This entire space is about to become part of the D$ array.
6. Now type PRINT PEEK 16400+256*PEEK 16401. As in Step 10 of Data Reduction, this answer is the first byte of D$. The 5 pointers following this address can be modified so as to consume the entire variables area of RAM. Record your answer.
7. Poke the 5 consecutive bytes beginning with your answer in Step 6 plus one.
   POKE 21342, PEEK 16434
   POKE 21343, PEEK 16435
   POKE 21344, 1
   POKE 21345, PEEK 16436-3
   POKE 21346, PEEK 16435
8. Cross your fingers again and type: PRINT LEN D$
9. In the Data Reduction routine, you had to clean up extraneous bytes left by shortening D$. In Data Expansion, you must reenter all of your other variables. When the array was enlarged, it literally consumed the other variables: P, Q$, E, X$, etc. All of the pointers, characters, and numbers are still in memory, but they’re not variables any more; they’re now part of D$. If you’re curious, type PRINT D$(10950 TO) to see what variables really look like. Re-enter the variable p. Type LET P=its old value given in step 3.
10. Then enter the remaining variables:
    LET QS$=””_ 32 spaces “
    LET C1=3
    LET C2=11
    LET S=1
    LET E=0
    LET Y$=” “
11. Type GOTO 17. You are now the proud owner of an enlarged data array and you didn’t have to retype a single file. Your old data is still there!

What Is Going On Here?

If all these RAND’s and POKE’s leave you cold, here is a different way to look at what’s going on in the data modification process. Think of an extension ladder. Each rung of the ladder is a byte of memory.

The D$ array is often considered to be a large contiguous block of memory much like you’d think of the whole ladder. But, the ladder could also be thought of as having 2 parts: the lower half and the upper extendable portion.

In data reduction, you take the 11000 character array (an 11000 foot ladder) and say, “This is really 2 separate arrays–one 8000 characters and the other 3000 characters in length.

The RAND’s and POKE’s are what do this. It’s like scratching out the label on the ladder that reveals its length. This may seem like six of one and half a dozen of another, but now comes the “coupe d’ grace”. We can release the extension-ladder and lower it down to make it one shorter ladder. This is carried over to Sinclair Basic when E$(3000) is redimmed to E$(1). In one fell swoop 3000 bytes are gone from the array. The memory space is available for other uses. Since it was E$ and not D$ that was redimmed, all the data in D$ remains untouched.

A ladder analogy can be used for data expansion too. In this case you must also take into account the program’s other variables (P, C1, S, Y$, Q$, etc.). Imagine a ladder that’s a little too short propped up against a building. This ladder is the initial D$(11000). Hanging from the top rung is a paint bucket which is full of the other variables used by the program.

Our worker nas painted as high. as he can reach. To go still higher he obtains a second ladder and secures it to the top of the first. This is what you do when you follow Step 4 of the expansion routine.

The paint bucket full of the program’s other variables which used to be at the top of the ladder is still there but it now is only midway up. The bucket is of little use until it is carried to the very top (done by Steps 9-10 in the expansion procedure).