How many files can I open in PB/dos 3.5?

[Basic Guy]

I'm working on a program that was going to open 15 or 16 files
simultaneously, but I got an error 67 (too many files). It works when I
have 14 or 15 files open, but it's the last file-open that is giving me
the error. The text of the error message says this:

"Caused either by trying to create too many files in a drive's root
directory, or by an invalid file name that affects the performance of
the DOS Create File system call".

The first condition does not really apply, but I know where it comes
from (back before FAT-32 when FAT-16 was limited to 512 files in the
root directory of a drive). That is not the situation here.

The second condition is also not happening (the file name is not
invalid).

The error condition says nothing about having too many open files (or
file handles), but I can't find any mention in PB/DOS about how many
files it can have open simultaneously. Or is that a DOS limitation?

BTW, I have "FILESHIGH=64" in my config.sys.

[Sjouke Burry]

Post by Basic Guy
I'm working on a program that was going to open 15 or 16 files
simultaneously, but I got an error 67 (too many files). It works when I
have 14 or 15 files open, but it's the last file-open that is giving me
"Caused either by trying to create too many files in a drive's root
directory, or by an invalid file name that affects the performance of
the DOS Create File system call".
The first condition does not really apply, but I know where it comes
from (back before FAT-32 when FAT-16 was limited to 512 files in the
root directory of a drive). That is not the situation here.
The second condition is also not happening (the file name is not
invalid).
The error condition says nothing about having too many open files (or
file handles), but I can't find any mention in PB/DOS about how many
files it can have open simultaneously. Or is that a DOS limitation?
BTW, I have "FILESHIGH=64" in my config.sys.

It has been as long as I can remember,DONT OPEN TO MANY FILES!!!!!
A dos limitation as well as a compiler limit.

[Richard Bonner]

Post by Sjouke Burry

Post by Basic Guy
I'm working on a program that was going to open 15 or 16 files
simultaneously, but I got an error 67 (too many files). It works when I
have 14 or 15 files open, but it's the last file-open that is giving me
"Caused either by trying to create too many files in a drive's root
directory, or by an invalid file name that affects the performance of
the DOS Create File system call".

*** Is this a DOS error message or a program message? If it's the
latter, check your documentation's "Troubleshooting" section to see if
this is mentioned. Perhaps there is a setting that will affect the file
handles.

Post by Sjouke Burry

Post by Basic Guy
The first condition does not really apply, but I know where it comes
from (back before FAT-32 when FAT-16 was limited to 512 files in the
root directory of a drive). That is not the situation here.
The second condition is also not happening (the file name is not
invalid).
The error condition says nothing about having too many open files (or
file handles), but I can't find any mention in PB/DOS about how many
files it can have open simultaneously. Or is that a DOS limitation?
BTW, I have "FILESHIGH=64" in my config.sys.

It has been as long as I can remember,DONT OPEN TO MANY FILES!!!!!
A dos limitation as well as a compiler limit.

*** Hmm, I have had way more than 15 files open on my DOS systems with
no problems. Might it be a memory issue?

[Klaus Meinhard]

Post by Richard Bonner
*** Hmm, I have had way more than 15 files open on my DOS systems
with no problems. Might it be a memory issue?

AFAIR DOS could handle up to 256 open file handles, for which mem was
reserved with the FILES=nn config.sys setting. And it was a memory issue
too: each handle used a few bytes. You can see that easily if you
experiment with the FILES=nn setting (FILESHIGH alleviated the problem
in later versions).

Filehandles were a real concern for data base proggies like Clipper
applications, who would need many open files _and_ a lot of available
mem.

[Happy Trails]

On Sun, 21 Mar 2010 09:49:17 +0100, "Klaus Meinhard"

Post by Klaus Meinhard

Post by Richard Bonner
*** Hmm, I have had way more than 15 files open on my DOS systems
with no problems. Might it be a memory issue?

AFAIR DOS could handle up to 256 open file handles, for which mem was
reserved with the FILES=nn config.sys setting. And it was a memory issue
too: each handle used a few bytes. You can see that easily if you
experiment with the FILES=nn setting (FILESHIGH alleviated the problem
in later versions).
Filehandles were a real concern for data base proggies like Clipper
applications, who would need many open files _and_ a lot of available
mem.

You could always trade off a little memory and time using FASTOPEN,
and just open and close those files needed for a sequence of
operations, making the actual number of files your programs could
process virtually unlimited.

I did a system once that was required to be able to update records in
any order in over 700 separate files. I tried running it with 50 - 100
files open simultaneously, but found that the transaction arrival
usually favoured no more than about 10 - 20 files within a minute or 2
time span, so I only opened 20 files at a time. I kept track of which
ones were open, and which ones had the most recent transactions
processed against them. If I needed a file that was not open, I'd
close one of the 20 open files - the one least recently used - and
open the one I needed. Since I was processing about 30 or more
transactions against a file when it got busy, and none when it wasn't,
this scheme worked very well. Using FASTOPEN meant I could close and
re-open files again very quickly - no directory read from disk was
needed.

[Richard Bonner]

Post by Happy Trails
On Sun, 21 Mar 2010 09:49:17 +0100, "Klaus Meinhard"

Post by Klaus Meinhard

*** ...I have had way more than 15 files open on my DOS systems
with no problems. Might it be a memory issue?

AFAIR DOS could handle up to 256 open file handles, for which mem was
reserved with the FILES=nn config.sys setting. And it was a memory issue
too: each handle used a few bytes. You can see that easily if you
experiment with the FILES=nn setting (FILESHIGH alleviated the problem
in later versions).

*** As I recall, each handle used 64 bytes of memory from MS-DOS 4
onward.

Post by Happy Trails

Post by Klaus Meinhard
Filehandles were a real concern for data base proggies like Clipper
applications, who would need many open files _and_ a lot of available
mem.

*** Yup.

Post by Happy Trails
You could always trade off a little memory and time using FASTOPEN,
and just open and close those files needed for a sequence of
operations, making the actual number of files your programs could
process virtually unlimited.

*** I think a better method is to employ a good disc cache with
aggressive settings. I use Hyperware's HyperDkx.

Post by Happy Trails
I did a system once that was required to be able to update records in
any order in over 700 separate files. I tried running it with 50 - 100
files open simultaneously, but found that the transaction arrival
usually favoured no more than about 10 - 20 files within a minute or 2
time span, so I only opened 20 files at a time. I kept track of which
ones were open, and which ones had the most recent transactions
processed against them. If I needed a file that was not open, I'd
close one of the 20 open files - the one least recently used - and
open the one I needed. Since I was processing about 30 or more
transactions against a file when it got busy, and none when it wasn't,
this scheme worked very well. Using FASTOPEN meant I could close and
re-open files again very quickly - no directory read from disk was
needed.

*** That would work, but given FASTOPEN's age, I would tend to believe
a newer disc cache would be more effective. HyperDkx is like using a
multitasker at times. It allows me to do other work while it writes to a
floppy in a very seamless manner for the most part.

[Happy Trails]

Post by Richard Bonner
*** That would work, but given FASTOPEN's age, I would tend to believe
a newer disc cache would be more effective. HyperDkx is like using a
multitasker at times. It allows me to do other work while it writes to a
floppy in a very seamless manner for the most part.

You have missed the point. No one has a problem with disk I/O delays -
it's the number of open files we are discussing here.

[Olav]

Hi,

Take a look in the example folder under the PB35 installation directory and
in the Dosunit.bas you will find a routine called AddFileHandles. Hopefully
calling this one will do the trick.
--
Olav

Post by Basic Guy
I'm working on a program that was going to open 15 or 16 files
simultaneously, but I got an error 67 (too many files). It works when I
have 14 or 15 files open, but it's the last file-open that is giving me
"Caused either by trying to create too many files in a drive's root
directory, or by an invalid file name that affects the performance of
the DOS Create File system call".
The first condition does not really apply, but I know where it comes
from (back before FAT-32 when FAT-16 was limited to 512 files in the
root directory of a drive). That is not the situation here.
The second condition is also not happening (the file name is not
invalid).
The error condition says nothing about having too many open files (or
file handles), but I can't find any mention in PB/DOS about how many
files it can have open simultaneously. Or is that a DOS limitation?
BTW, I have "FILESHIGH=64" in my config.sys.

[Zaphod Beeblebrox]

Post by Basic Guy
I'm working on a program that was going to open 15 or 16 files
simultaneously, but I got an error 67 (too many files). It works when I
have 14 or 15 files open, but it's the last file-open that is giving me
"Caused either by trying to create too many files in a drive's root
directory, or by an invalid file name that affects the performance of
the DOS Create File system call".
The first condition does not really apply, but I know where it comes
from (back before FAT-32 when FAT-16 was limited to 512 files in the
root directory of a drive). That is not the situation here.
The second condition is also not happening (the file name is not
invalid).
The error condition says nothing about having too many open files (or
file handles), but I can't find any mention in PB/DOS about how many
files it can have open simultaneously. Or is that a DOS limitation?
BTW, I have "FILESHIGH=64" in my config.sys.

I ran into a situation some time ago similar to this where my program
would error opening the 16th file. In my case it was because I wasn't
closing the files when I thought I was, but I did some research to
find out why the FILES= setting wasn't preventing the error. What I
found out was that there is a limit to how many files can be open by a
single program / process that is different than the FILES= limit,
which is system-wide. A single process is limited to 20 open files,
and DOS uses up 5 of those right off the top for CON (stdin, stdout,
and stderr), AUX (stdaux), and PRN (stdprn), leaving you 15 to play
with. There is a way to change this with a call to INT 21 AH=67, Set
Handle Count (but you still are limited by the FILES= setting so keep
that in mind.) See the comp.os.msdos.programmer FAQ part 3/5 at
http://www.faqs.org/faqs/msdos-programmer-faq/part3/section-12.html