What happens when one thread closes a file while other threads are still writing

I have been thinking about this question for close to three months, and it was probably during one of those nights when my mind wandered into systems internals that I finally found the answer.

This article is an account of all the possibilities I thought of and put together after substantial research on the subject.

Picture this;

fd = open("file.txt", O_CREAT | O_RDWR | O_APPEND, 0644);

buf = "Hello world";
buf_size = strlen(buf);

/*Thread A enters write*/
write(fd, buf, buf_size)

/*Thread B concurrently writes to the same file*/
write(fd, buf, buf_size)

/*Thread C closes this file; it might be*/
close(fd)

Imagine thread C calls close() while threads A and B are still executing inside the kernel as part of their write() system calls.

My first intuition was that once thread C successfully closes the file descriptor, the kernel would invalidate it immediately. If threads A and B were still in the middle of their write() calls, surely those writes would fail with EBADF, right?

Though after further research, it led me to drop my assumptions.

Let’s take a detour and see a skeleton of how a write system call is implemented in the kernel.

SYSCALL_DEFINE3(write, unsigned int, fd, const char __user *, buf, size_t, count)
{
    ...
    /* 1. Get struct file from fd */
    struct fd f = fdget_pos(fd);

    /* 2. Get current position */
    loff_t pos = file_pos_read(f.file);

    /* 3. Dispatch to VFS */
    ret = vfs_write(f.file, buf, count, &pos);
    
    ...
}

Step 1 does something interesting; gets a struct file from the passed file descriptor. A file descriptor is a non-negative integer that is an index in the process’ file descriptor table. Each entry in that table maps to an open file description, represented in Linux by a struct file.

The order of these operations isn’t accidental. In fact, the entire answer to our question hinges on why Linux performs them in this sequence.

First, the kernel looks up the descriptor, it then retrieves the struct file object, and finally increments its reference count. At this point, going into vfs_write, the system call no longer depends on the file descriptor itself. It holds its own reference to the underlying struct file, ensuring that the object remains alive even if another thread closes the descriptor.

This led me to the realization that a call to close() doesn’t physically close or delete a file. Instead, it removes one reference to the underlying open file description (struct file). Only when the last reference to that struct file is released does the kernel releases the underlying struct file, destroying that open file description.

Scenario 2;

fd = open("file.txt", O_CREAT | O_RDWR | O_APPEND, 0644);

buf = "Hello world";
buf_size = strlen(buf);

/*Thread A successfully writes to the file and returns the number of bytes written*/
write(fd, buf, buf_size)

/*Thread B closes the file*/
close(fd)

/*Thread C tries to write to the file while thread B is closing the file*/
write(fd, buf, buf_size)

Assuming thread C enters the kernel after thread B has completed the descriptor lookup removal, the write() fails with EBADF. At that point, the file descriptor no longer exists in the process’s file descriptor table, so the kernel cannot resolve it to an open file description.

Thread B, upon entry into the kernel will remove the file descriptor fd from the file descriptor table. This is the reason why thread C fails with an EBADF since it cannot resolve an open file description tied to the file descriptor.

There is one final subtlety that’s worth mentioning. Although the kernel keeps the struct file alive until all in-flight operations complete, it does not reserve the file descriptor number itself.

Once close(fd) removes the descriptor from the process’s file descriptor table, that integer is immediately available for reuse. A subsequent open() may return the same descriptor number while another thread is still finishing a write() that began before the close().

Something to think about;

What would happen if thread A opens file A and receives descriptor 3. Thread B later closes descriptor 3. Thread C opens file B, and the kernel immediately reuses descriptor 3 for that file. Meanwhile, thread D, still believing that descriptor 3 refers to file A, issues a write(3, …). Which file receives the write?