Skip to content

Conversation

@marcelfarres
Copy link

I was trying to create folders & files in the SD card and it was not working.

This changes seem to make it functional again.

I test it on a M4 Gran Central, and seems to have dir persistent and creation even after mounting/un-moutning the file system.

Not an expert in any extend on TinyGo, hope it helps/you can test it/review it.

  • Add Sync() after Format, Unmount, Remove, Rename, Mkdir
  • Add Sync() after File.Close, File.Sync, File.Truncate
  • Sync on OpenFile with O_CREATE/O_TRUNC flags
  • Fix cache invalidation in lfs_bd_flush
  • Add TestDirectoryPersistence and NestedDirectories tests"

Should fix #9

- Add Sync() after Format, Unmount, Remove, Rename, Mkdir
- Add Sync() after File.Close, File.Sync, File.Truncate
- Sync on OpenFile with O_CREATE/O_TRUNC flags
- Fix cache invalidation in lfs_bd_flush
- Add TestDirectoryPersistence and NestedDirectories tests"
@marcelfarres
Copy link
Author

Fix filesystem persistence on SD cards

Problem

I was working with LittleFS on a Grand Central M4 with an SD card and noticed that directories and files I created weren't surviving a power cycle. I'd run mkdir, it would succeed, ls would show the directory. But after unplugging and reconnecting (or even just unmounting and remounting), everything was gone.

Investigation

I traced the data flow through the code:

  1. When I call fs.Mkdir("test"), it goes to go_lfs.go:247
  2. That calls the C library's lfs_mkdir() via CGo
  3. LittleFS writes through the callbacks in go_lfs_callbacks.go to the SD card driver
  4. The SD card driver buffers the writes... and that's where they stay

The problem: the SD card driver holds writes in a buffer for efficiency. Without explicitly calling Sync(), that buffer never gets flushed to the physical flash. On power loss, the buffered data is gone.

Looking at the original code, I noticed that mutating operations just returned after calling the C function—no sync. The block device interface in tinyfs.go defines a Syncer interface, but it wasn't being used after filesystem operations.

What I changed

littlefs/go_lfs.go

Added Sync() calls after every operation that modifies the filesystem. Example for Mkdir:

func (l *LFS) Mkdir(path string, _ os.FileMode) error {
    cs := (*C.char)(cstring(path))
    defer C.free(unsafe.Pointer(cs))
    if err := errval(C.lfs_mkdir(l.lfs, cs)); err != nil {
        return err
    }
    if syncer, ok := l.dev.(tinyfs.Syncer); ok {
        return syncer.Sync()  // <-- This was missing!
    }
    return nil
}

Same pattern applied to:

  • Format() - sync after formatting
  • Unmount() - sync before closing
  • Remove() - sync after deletion
  • Rename() - sync after rename
  • OpenFile() with O_CREATE/O_TRUNC - sync after creating
  • File.Close() - sync after closing
  • File.Sync() - sync to block device
  • File.Truncate() - sync after truncation

littlefs/go_lfs_test.go

Added tests:

  • TestDirectoryPersistence - creates a directory, unmounts, remounts, verifies it persists
  • NestedDirectories - creates 3 levels of nested directories

Testing

Tested on actual hardware (Grand Central M4 + SD card):

  • Formatted the card
  • Created nested directories (level1/level2/level3)
  • Created files
  • Unmounted and remounted
  • Everything persisted ✓

return nil, err
}

if flags&(os.O_CREATE|os.O_TRUNC) != 0 {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not think it makes sense to call Sync() on an Open().

if err := errval(C.lfs_file_sync(f.lfs.lfs, f.fileptr())); err != nil {
return err
}
if syncer, ok := f.lfs.dev.(tinyfs.Syncer); ok {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually Sync() on a Sync()? Seems like a yes! 👍

@deadprogram
Copy link
Member

Hello @marcelfarres thanks for the code submission. Please see my comments.

@deadprogram
Copy link
Member

I probably did not notice the issue due to using flash pretty much all of the time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants