Invisible files associated with OS X keychains

Question or issue on macOS:

It seems that a keychain file (with extension .keychain) will usually have an invisible file associated with it, located in the same directory.

This invisible file always has these properties:

The invisible file can be deleted, but when the keychain’s contents are next modified, the invisible file will be recreated with the same name.

Here are some steps to demonstrate, using the Keychain Access utility:

I’ve verified that this occurs in OS X 10.8 and 10.9.

The same invisible files are created when manipulating keychains using Apple’s security tool in the Terminal:

How to solve this problem?

After a lot of investigating, I’ve managed to answer most of my questions:

1. The invisible file implements a write lock for the keychain’s database.
2. .fl is the filename prefix for lock files created by the AtomicFile class in the Security framework.
3. The 8 hex characters in the filename are the beginning of the SHA-1 hash of the keychain’s filename. For example, if the keychain file is called Test.keychain, then the SHA-1 hash of its filename begins with 1BCE4B9A... and so the lock file will be called .fl1BCE4B9A.
4. I haven’t discovered a way to prevent the lock file from being created when a keychain is created or modified. I think it’s probably impossible, but I’d be very interested if someone can figure out a way to do it.

---

Here are the details of my investigation:

Keychain’s locked/unlocked status

I noticed that the invisible file is not affected by the keychain’s locked/unlocked status. If the invisible file has been deleted, then locking/unlocking the keychain does not recreate the invisible file.

System calls

I did some investigating using the File Activity template in Apple’s Instruments tool.

These system calls are responsible for manipulating the invisible file:

• Creating the invisible file when a new keychain is created:
  • Security::AtomicFile::create(unsigned short)
  • Security::RefPointer<Security::AtomicLockedFile>::release_internal()
  • Relevant part of the call tree
• Recreating the invisible file when a keychain’s contents are modified:
  • Security::AtomicFile::write()
  • Security::RefPointer<Security::AtomicLockedFile>::release_internal()
  • Relevant part of the call tree
• Deleting the invisible file when a keychain is deleted:
  • Security::AtomicFile::performDelete()
  • Relevant part of the call tree

C++ files

These are the relevant files and classes (source code available from Apple Open Source for OS X 10.9.2):

• AtomicFile.cpp
  • Security::AtomicFile
  • Security::AtomicLockedFile
  • Security::AtomicTempFile
  • Security::LocalFileLocker
• AppleDatabase.cpp
  • Security::AppleDatabase
  • Security::DbModifier

Comments in source code

The comments in those files provided some clues:

• AtomicFile::AtomicFile()
  • “compute the name of the lock file for this file”
• AtomicFile::create()
  • “Lock the file for writing and return a newly created AtomicTempFile.”
  • “Now that we have created the lock and the new db file create a tempfile object.”
• LocalFileLocker::lock()
  • “if the lock file doesn’t exist, create it”
  • “try to get exclusive access to the file”
  • “check and see if the file we have access to still exists. If not, another file shared our file lock due to a hash collision and has thrown our lock away — that, or a user blew the lock file away himself.”
• DbModifier::modifyDatabase()
  • “Now we are holding the write lock”
• AtomicFile::write()
  • “Lock the database file for writing and return a newly created AtomicTempFile.”
• AtomicFile::performDelete()
  • “Aquire the write lock and remove the file.”
  • “unlink our lock file”

Generation of lock file’s name

I found this code in the AtomicFile constructor:

char buffer[256];
sprintf(buffer, "%08X", hash);
mLockFilePath = mDir + ".fl" + buffer;

where hash is the first 4 bytes of the SHA-1 hash of the keychain’s filename.

Note: using only 4 bytes (32 bits) of the hash, there’s a reasonable chance of a hash collision, which is mentioned in the comment in LocalFileLocker::lock().

Operation of lock

The flock() system call is used to manipulate the lock on the lock file.

Here’s the call tree when the keychain’s database is being locked for writing:

DbModifier::createDatabase() or ::modifyDatabase() or ::deleteDatabase()
  AtomicFile::create() or ::write() or ::performDelete()
    AtomicLockedFile::lock()
      LocalFileLocker::lock()
        flock(mLockFile, LOCK_EX)  // exclusive lock

and when it’s being unlocked after writing:

DbModifier::commit()
  AtomicTempFile::commit()
    RefPointer::setPointer(AtomicLockedFile*)
      RefPointer::release_internal()
        AtomicLockedFile::~AtomicLockedFile()  // destructor
          AtomicLockedFile::unlock()
            LocalFileLocker::unlock()
              flock(mLockFile, LOCK_UN)  // unlock

Hope this helps!