虚拟文件系统

Linux*VirtualFileSystemIntel®ChinaSoftwareCenter*OthernamesandbrandsmaybeclaimedasthepropertyofothersAgendaLinux*filesystemoverviewVFSdatastructuresPathlookupSystemcallsAsimpleFSimplementation*Othernamesandbrandsmaybeclaimedasthepropertyofothers2024/1/172Linux*filesystemsVirtualFileSystemlikeotherUnixLinux*supportsmanyfilesystemtypesEachtypeisadifferentfilesystemimplementationUsuallyinaloadablemoduleLinux*supportsmanyfilesysteminstancesEachinstanceisamountedfilesystemEverysystemhasarootfilesystem/………/………hd0hd1*Othernamesandbrandsmaybeclaimedasthepropertyofothers2024/1/173Linux*kernelfilesystemarchitectureVFSlayerext3ntfsvfatnfssmbprocsysfsBlockdevicelayernetworklayerGenericfilesysteminterface*Othernamesandbrandsmaybeclaimedasthepropertyofothers2024/1/174Linux*filesystemtypesBlockdevice-basedfilesystemsBuildontopofablockdeviceExt3,ntfs…Network-basedfilesystemsBuildontopofnetworkprotocolsNfs,smb…Virtual/pseudofilesystemsBuildinmemoryProc,sysfs…*Othernamesandbrandsmaybeclaimedasthepropertyofothers2024/1/175VFSlayerObject-orientedUnifiedfilesystemmodelTraditionalUnixlikeFSimplementationshouldbeconvertedtothemodelifnecessaryPurposeUnifieduserinterfaceAbstractionforalldifferentfilesystemimplementationsResponsibilitiesSystemcallsManagefilesystemsdatastructuresInteractwithdifferentfilesystemimplementations2024/1/176VFSobjectsImportant:harddiskVS.kernelSuperblockInfoaboutafilesysteminstanceMayhavediskcounterpartInodeInfoaboutafileMayhavediskcounterpartDentryaboutdirectorytreeoraboutanamepathFileaboutanopenedfile2024/1/177DiskVSkernelsuperblockinodeblocksdatablocksdentryinodefilesuperblockdentrydentryinodekerneldisk2024/1/178VFSobjectsDiskfilesuperblockobjectInodeobjectdentryobjectdentryobjectfileobjectfileobjectprocess1process2process3fileobjectfdf_dentryd_inodei_sb2024/1/179VFSobjectscommongroundReferencecounterforeachobjectShare(openanobjectseveraltimes)StatetransitionCacheCacheunusedobjectsinmemoryForperformanceconsiderationAbstractobjectsTheobjectmightnotequaltothecorrespondingoneofspecificfsimplementation(disk)SpecificfsimplementationshouldconverttheirdiskonetokerneloneObjectanditsoperationsForeachobjectClassicalobject-oriented2024/1/1710VFSobjectmethodsAnoperationtableforeachobjectFunctionpointersFilesystemspecificormaybeobjectspecificLinkVFSwithspecificfilesystemimplementationImplementedbyspecificfilesystemCalledinVFS2024/1/1711SummaryConcepts:VFS,fsroot,systemroot…ObjectorientedmethodVFSobjects:diskvs.kernel2024/1/1712AgendaLinux*filesystemoverviewVFSdatastructuresPathlookupSystemcallsAsimpleFSimplementation*Othernamesandbrandsmaybeclaimedasthepropertyofothers2024/1/1713FilesystemtypeAboutafilesystemimplementationstruct

file_system_typeMultiplefsinstancesperfile_system_type

MainfieldsName(fsname,like‘ext3’)Fs_flags(flags,like‘FS_REQUIRES_DEV’)Get_sb(calledwhenmountinganewfsinstance)Kill_sb(calledwhenunmountingafsinstance)Fs_supers(linktoalistforallfsinstancesofthetype)Owner(themoduleholdingthefsimplementation)2024/1/1714FilesystemtypesnameGet_sbKill_sbnextFs_supersnameGet_sbKill_sbnextFs_superss_instancess_instancess_instancesfile_systemsstruct

fs_system_typestructsuper“ext3”ext3_get_sb2024/1/1715RegisterafilesystemFilesystemimplementationmustregister/unregisteritintokernelUsuallyinamoduleinit/exit!=mountafilesystemregister_filesystemunregister_filesystemget_sb(superblockreadingfunction)isthekey2024/1/1716Superblock

objectInformationforafsinstanceStruct

superblockMaybeindisk.KernelhasacachedoneforaninstanceMainfieldsS_bdev–blockdeviceS_type–filesystemtypeS_root–dentryofrootdirectoryS_op–superblockoperationsS_inodes,s_dirty,s_io–inodelistsS_blocksize,s_maxbytes–parametersS_magic–magicnumberofthefsS_count–referencecount…2024/1/1717Superblockoperationsstruct

super_operationswrite_super(readsuperisinfilesystemtype)read_inode/write_inode(read/writeaninodefrom/todisk)dirty_inode(makesaninodedirty,soitcanbewrittenbacktodisk)delete_inode(deletetheinodefromdisk)…2024/1/1718InodeobjectInoderepresentsafile,storefile’sinfoInoderesidesindisk,andkernelhasacachedoneInodenumberisuniqueinonefsinstanceInodecache.Recentlyaccessedunusedinodewillbecachedforperformancepurpose2024/1/1719InodestructureStruct

inodeMainfieldsi_ino,i_sb(inodenumber&superblock)i_mapping(addressspace)i_fop(Fileoperations)i_op(Inodeoperations)i_list,i_hash,i_sb_list(lists)i_uid,i_gid,i_mod,i_atime(attributes)i_count(referencecount)2024/1/1720Inodelistsi_listinode_in_use(inusebutnotdirty,i_count>0)inode_unused(unsed,i_count=0.unusedinodeisn’tfreedimmediately)super_block->s_dirty(dirty,i_count>0)i_sb_listsuper_block->s_inodesi_hashFastsearchacachedinodeinode_hashtableHashedbyi_ino&i_sb2024/1/1721Inodelistsi_hashi_listi_hashi_listi_hashi_listinode_in_useinode_unuseds_dirtystruct

super_blockInode_hashtablestruct

inodes_inodesi_hashi_listi_sb_listi_sb_listi_sb_listi_sb_list2024/1/1722Inodeoperationsstruct

inode_operationslookup(lookupaninodeinaparentdir)create(createanewfile)mkdir/rmdir(create/deletedir)Link/unlink/symlink(createhardlink/symbollink)…InodekernelAPIsIget(),ilookup(),…2024/1/1723DentryobjectKerneldatastructurefordirectoryentryKernelcache,nodiskcounterpartFilenametoinodemappingMultipledentriesmaypointtooneinodeDentrycache.Recentlyaccessedununsed

dentrywillbecachedforperformancepurposeExample:/xxx/yyy(3dentries,‘/’,‘xxx’,‘yyy’)2024/1/1724DentrystructureStruct

dentryMainfieldsD_inode(its

inode)D_parent,d_child,d_subdirs,d_alias,d_lru(links)D_nameD_mounted(iftherearefsmounted)D_op(dentryoperations)…2024/1/1725DentrystatesfreeInused_aliaslinkstoinode->i_dentryUnusedd_lrulinkstodentry_unusedNegativeNoinode2024/1/1726DentrylistTreelayout(d_parent,d_subdirs,d_child)Inuse

list(d_alias)Unusedlist(d_lru)Hashtable(hashcollisionlist,d_hash)2024/1/1727Dentrytreelayoutd_parentd_childd_subdirsd_parentd_childd_subdirsd_parentd_childd_subdirs2024/1/1728Dentrylistsd_inoded_hashd_lrud_aliasd_inoded_hashd_lrud_aliasd_inoded_hashd_lrud_aliasd_inoded_hashd_lrud_aliasdentry_unusedi_dentrystruct

inodestruct

dentrydentry_hashtable2024/1/1729DentryoperationsDentry_operationsd_hash(returnhashvalueofafilename)d_compare(compareiffilenameanddentrymatch)d_delete(deleteadentry,whend_count=0)…DentrykernelAPIsd_add(),d_alloc(),d_lookup()…2024/1/1730FileobjectDescribeshowaprocessinteractswithanopenedfileResidesinkernelStructfilef_op(fileoperations)f_dentry(file’sdentry)f_uid/f_gid/f_mode/f_owner(fileattributes)fu_list(linktosuperblock,remountshouldcheckopenedfiles)f_pos(read/writeposition)f_count(referencecount)…2024/1/1731FileoperationsStruct

file_operationsOpenwrite/readllseek(setfileread/writeposition)mmap(calledwhendoingmmap

syscall)release(calledwhenclosingafile)poll(calledwhendoingselect/pollsyscall)ioctl…2024/1/1732VFSobjectsrelationships_roots_inodess_filesi_sbi_sb_listi_dentryd_aliasd_sbd_inodefu_listf_dentrystruct

super_blockstruct

inodestructfilestruct

dentry2024/1/1733StructuresassociatedwithaprocessStruct

task_structhastwofieldsrelatedwithfsstruct

fs_struct*fsstruct

files_struct*filesstruct

files_structAboutopenedfilesofataskImportantfields:Structfilearray(fd<->filemapping)Togetanopenfile:files_struct->fd[i],iisthefilehandlerStruct

fs_structAbouttask’sfilesystemenvironmentRootmountpoint(vfsmount),rootdentryPwd

mountpoint(vfsmount),pwd

dentry2024/1/1734SummaryObjectsandtheirdatastructures/operationsReferenceKernelsrc/include/fs.hKernelsrc/fs/filesystems.c(struct

file_system_type)Kernelsrc/fs/super.c(structsuper)Kernelsrc/fs/inode.c(struct

inode)Kernelsrc/fs/dcache.c(struct

dentry)2024/1/1735AgendaLinux*filesystemoverviewVFSdatastructuresPathlookupSystemcallsAsimpleFSimplementation*Othernamesandbrandsmaybeclaimedasthepropertyofothers2024/1/1736PathnamelookupFindtheinode/dentryfromagivenpathnameFrequentlyusedStartpointdentryPathnamestarts‘/’:fs->rootElse:fs->pwdSpecialhandlingSymboliclinks(checkloops)MountpointsAccesspermission2024/1/1737Path_walkGetpath’sstartpoint(root,pwd)anditsmountpointParseonenameoneloopCheckaccesspermission(vfs_permission)CalculatenamehashCheckspecialfile(eg.“.”)Checkdentrycache(__do_lookup)Orreadinodefromdisk(real_lookup)Checkmountpoint,symboliclink2024/1/1738Lookupindcache(dentrycache)__do_lookupGethashcollisionlist(d_hash,parentandfilenameashashkey(parent->d_op->d_hash()))ScanthelisttogetonedentryParent->d_op->d_compare(),matchfilename2024/1/1739LookupindiskWhendentrycachelookupfailsreal_lookupAllocateadentry(d_alloc())Parent->i_op->lookup(),whichreadsinodefromdisk.Thisisafsspecificimplementation.2024/1/1740Lookupexamplestruct

dentry*ext3_lookup(structinode*dir,struct

dentry*dentry,struct

nameidata*nd){ …/*getinodenumber*/

inode=iget(dir->i_sb,ino); … returnd_splice_alias(inode,dentry);}2024/1/1741Lookupexampleiget(Getinodeofinodenumber)SearchinodecacheOrallocateanewinode(sb->s_op->alloc_inode(),filesystemspecific)Readinodein(sb->s_op->read_inode(),filesystemspecific)d_splice_alias(linkdentrywithinode)Adddentrytoinode’slistAdddentryintohashlist(soitcanbefoundbycachedlookup)2024/1/1742PathwalkspecialhandlingMountpoint–follow_mountFsmountisstackedIt’spossibletomountmultiplefsinthesamedirectoryjumptothelatestmountedfilesystemonthedentry‘..’–follow_dotdotSymboliclinkSpecificfsmustsupportsymboliclinkDo_follow_link(checkloop,jumptolinkedfile.Usesinodeoperation’sfollow_link(),put_link())2024/1/1743PathwalkexampledentryisfoundatndwithasuccesspathwalkExample

struct

nameidata

nd;

path_walk(name,&nd); …/*dosomethingwithnd->dentry*/

path_release(&nd);2024/1/1744SummaryHowkernelparsesafilenameTheexamplehowkernelusesobjects’operationsReference:Kernelsrc/fs/namei.c2024/1/1745AgendaLinux*filesystemoverviewVFSdatastructuresPathlookupSystemcallsAsimpleFSimplementation*Othernamesandbrandsmaybeclaimedasthepropertyofothers2024/1/1746MountafilesystemMountingafilesystemInitializeafsinstanceGrafttherootofafstorootfs’sleafSpecialfsinstancesmightnotbeinrootfsTerminologiesMountpointRootdirectoryofamountedfsRootfs2024/1/1747Struct

vfsmountMainfieldsMnt_root(rootdentryofthemountedfs)Mnt_sb(superblockofmountedfs)Mnt_mountpoint(dentryofmountpoint)Mnt_parent(parent

vfsmount)Mnt_hash,mnt_child,mnt_mounts(variantlists)2024/1/1748Vfsmountlistsmnt_mountsmnt_childmnt_hashmnt_mountsmnt_childmnt_hashmnt_mountsmnt_childmnt_hashmount_hashtable(parent_vfsmount,mountpoint_dentry)strutvfs_mount2024/1/1749sys_mountsys_mountpath_lookup(getmountpoint

dentry)do_kern_mount(initializethefsinstance)Allocatevfsmountandfillitfile_system_type->get_sb()getsuperblockdo_add_mount(grafttheinstanceintoroottree)Followdownmountpoint(adentrycanmountmultiplefs)graft_tree(addvfsmounttoproperlists)2024/1/1750Opensystemcallsys_openGetanemptyfdPathwalktogetthedentryofafilename(open_namei)OpenthedentryGetafilestructandinitializeitperitsdentry(suchasinitializing‘fops’)Callfops->openfd_install(link‘fd’with‘file’)2024/1/1751Struct

address_spacepresentsafile’spagesLinkProcess<->address_spaceAddress_space<->pageStruct

address_spacestruct

inode *host;struct

radix_tree_root

page_tree;struct

prio_tree_root

i_mmap;struct

address_space_operations*a_ops;Struct

address_space_operations.read_page/.write_page.set_page_dirty2024/1/1752Readsystemcallsys_readGetfilestructperfd(fget_light)Checkaccesspermissionfile->f_op->read()Updatefileread/writepositioncounterread/writedatacache(buffer)forperformancef_op->readdoestherealfilereadCoulddirectreaddiskhereButusuallyreliesonaddress_space’s.read_pagemethod.Kernelprovideshelperfunctions(generic_file_read)whichhavebufferhandlinggeneric_file_read(abbreviatedsteps)SanitycheckCheckifthedataisinpagecache,ifyes,jumptolaststepReadpagein-Address_space->a_ops->read_page()AddreadpageintopagecacheCopydatatouserspace2024/1/1753Writesystemcallsys_writeSimilarstepsassys_readfile->f_op->write()dorealfilewritefile->f_op->write()CouldwritedatatodiskdirectlyKernelhelpergeneric_file_writegeneric_file_writeKernelwilldobuffer,usuallyjustcopythedatafromuserspacetokernelandmarkthepageanditsinodedirty.Reliesonaddress_space->a_ops->prepare_write()(preparebuffer,readsomedatain…)andaddress_space->a_ops->commit_write()(dirtydata)Dirtypage(a_ops->dirty_page())Dirtyinode(superblockoperation’sdirty_inode())Pagewritebackwritesthechangestodiskfinally.Physicallywriteinodetodisk(superoperation->write_inode())Physicallywritepagetodisk(a_ops->write_pages/write_page())2024/1/1754SummaryDatastructuresaboutmountpointImplementationoftypicalfsrelatedsystemcallsHowkernelusesobjects’operationsReferenceKernelsrc/fs/namespace.cKernelsrc/fs/open.cKernelsrc/fs/read_write.c2024/1/1755AgendaLinux*filesystemoverviewVFSdatastructuresPathlookupSystemcallsAsimpleFSimplementation*Othernamesandbrandsmaybeclaimedasthepropertyofothers2024/1/1756AsimpleFSimplementation-romfsAreadonlyfilesystemimplementationVerysimple(lessthan1000linescode)FSformatdocumentcanbefoundat(kernelsource/Documentation/filesystems/romfs.txt)Codecanbefoundat(kernelsource/fs/romfs)2024/1/1757Romfs–file_system_typestaticstruct

file_system_type

romfs_fs_type={ .owner =THIS_MODULE, .name ="romfs", .get_sb =romfs_get_sb, .kill_sb =kill_block_super, .fs_flags =FS_REQUIRES_DEV,};staticint__initinit_romfs_fs(void){ err=register_filesystem(&romfs_fs_type);}module_init(init_romfs_fs)2024/1/1758Romfs–file_system_typeromfs_get_sbUseshelperroutine(get_sb_bdev)toopenblockdevice,allocatesuperblockstructandinitializeitGetsector0(sb_bread)fromblockdevice.Sector0hassuperblockinfodosanitycheck(magicnumber…)getrootinode’s

inodenumber(insuperblock)Getinode

struct(iget)s_op->read_inode()toreadinodeinInitializerootdentry“/”(d_alloc_root)kill_block_superCommonhelperroutineCloseblockdev,freeresources,writedat