Skip to content

vita: tell stdio what to buffer a file with - #116

Open
frangarcj wants to merge 1 commit into
vitafrom
vita-stdio-blksize
Open

vita: tell stdio what to buffer a file with#116
frangarcj wants to merge 1 commit into
vitafrom
vita-stdio-blksize

Conversation

@frangarcj

Copy link
Copy Markdown
Contributor

vita: tell stdio what to buffer a file with

What does this change?

Fills in st_blksize for regular files, and defines HAVE_BLKSIZE for the
target so newlib's stdio consults it.

Two lines of effect: configure.host gains -DHAVE_BLKSIZE on the arm*vita*
entry, and scestat_to_stat sets st_blksize to 64 KiB when the file is a
regular file.

Why is this change needed?

__swhatbuf_r asks fstat for st_blksize and uses it as the stdio buffer
size, falling back to BUFSIZ when it gets nothing. On Vita it got nothing
twice over: the target never defined HAVE_BLKSIZE, so the branch that reads
the field is not even compiled, and nothing in sys/vita ever filled the field
in. Every FILE therefore got BUFSIZ, which is 1024.

That is not merely a small buffer. It is slower than switching buffering off:

stdio buffer MB/s
none 6.99
1 KiB (today) 2.64
4 KiB 6.97
8 KiB 8.67
16 KiB 9.93
32 KiB 10.74
64 KiB 11.16
128 KiB 11.40
256 KiB 11.53

Reading a file in 4 KiB fread calls, the caller asks for 4 KiB and the buffer
holds 1, so each call becomes four device reads plus a copy through the buffer.
Unbuffered at least passes the request straight down.

The ceiling for the device is 11.78 MB/s, from sceIoRead in 256 KiB chunks;
the same read in 64 KiB chunks gives 11.38. Buffered stdio tracks raw sceIoRead
almost exactly at matching sizes, so the buffer size is the whole story here.
64 KiB reaches 95% of that ceiling and is where the curve flattens — 128 KiB
buys another 2%, 256 KiB another 1%. It is also what SceLibc uses, which its
setvbuf and setbuf show as a hardcoded 0x10000.

The device cannot be asked for its real cluster size. sceIoDevctl with 0x3001,
which is what statvfs uses to fill f_bsize, fails from a normal application
for every spelling of the drive: 0x80010030 for ux0: and ux0:/,
0x80010016 for ux0, 0x80010013 for ux0:data. So the value has to be a
constant.

Testing

Measured on real PlayStation Vita hardware with a purpose-built probe: it writes
an 8 MiB file, then reads it back in 4 KiB fread calls with the stdio buffer
set through setvbuf to each size above, reopening the file for every pass and
taking the best of three. It also measures sceIoRead directly at 4, 16, 64 and
256 KiB for the ceiling, and reports what the device answers to the 0x3001
devctl. The numbers above are that run.

Compatibility

No API or ABI change. st_blksize is an existing field that was always left at
zero.

Two behaviour changes worth a reviewer's attention:

  • A FILE opened on a regular file now allocates a 64 KiB buffer instead of
    1 KiB. A program holding 20 open files goes from 20 KiB to 1.25 MB of stdio
    buffers. Callers that care can still choose with setvbuf, and 32 KiB reaches
    91% of the ceiling for half the memory if that trade is preferred.
  • A caller reading st_blksize itself now gets 65536 rather than 0 for regular
    files. Anything sizing a buffer from it was previously getting zero.

Directories, ttys, sockets and pipes are untouched and keep falling back to
BUFSIZ, so a process that only prints does not pay for stdout.

No RFC required.

Third-party material

None.

AI assistance

  • Claude Opus 5

Additional context

The same devctl failure means statvfs returns -1 with EIO on hardware for
any path, since fs.c builds f_bsize and the block counts from that call.
sceAppMgrGetDevInfo answers correctly where the devctl does not. That is a
separate fix and not part of this pull request.

__swhatbuf_r asks fstat for st_blksize and falls back to BUFSIZ when
nobody answers.  Nothing here ever filled it in, and configure.host did
not define HAVE_BLKSIZE for the target either, so every FILE got 1024
bytes.  Reading a file in 4 KiB calls measured 2.64 MB/s that way against
6.99 with no buffer at all: the caller asks for 4 KiB, the buffer holds
one, so each read becomes four device reads plus a copy through it.

Measured on hardware, MB/s, against a 11.78 ceiling from sceIoRead:
1 KiB 2.64, 4 KiB 6.97, 16 KiB 9.93, 32 KiB 10.71, 64 KiB 11.12,
256 KiB 11.49.  SceLibc uses 64 KiB and the curve flattens there.

The device cannot be asked for its cluster size: sceIoDevctl 0x3001,
which is what statvfs uses, returns 0x80010030 from a normal application
for every spelling of the drive.

Only regular files get it, so a process that only prints keeps its tty on
the old 1024 rather than paying 64 KiB for stdout.

Assisted-by: Claude Opus 5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant