#LyX 1.3 created this file. For more info see http://www.lyx.org/
\lyxformat 221
\textclass article
\begin_preamble
\usepackage{listings}
\usepackage[usenames]{color}
\usepackage{courier}

%\usepackage{pt}
%\usepackage{units}
%\usepackage{coz}

%\usepackage{epsf}
%\usepackage{lncsexample}
%\conttrue
%\usepackage{times}
%\usepackage{top}
%\usepackage{graphicx}
\newcommand{\lst}[3] {
        \noindent\vspace{-1mm}
        \definecolor{cKeyword}{rgb}{0.8,0.1,0.1}
        \definecolor{cComment}{rgb}{0.2,0.5,0.7}
        \definecolor{cString}{rgb}{0.2,0.7,0.2}
        \lstinputlisting[caption={#2},
        label={#1},
        showstringspaces=false, 
        numbers=left, 
        stepnumber=1,
        frame=lines,
        language=C,
        extendedchars=true,
        basicstyle=\small\tt,
        numberstyle=\tiny,
        keywordstyle=\color{cKeyword},
        commentstyle=\color{cComment},
        stringstyle=\color{cString},
        directivestyle=\color{magenta},
        emph={1, 2, 3, 4, 5, 6, 7, 8, 9, 0, NULL, lustre, CFS},
        emphstyle=\color{blue},
        breaklines=true,
        #3]{#1}\vspace{0.3mm}
}
\end_preamble
\language english
\inputencoding koi8-r
\fontscheme times
\graphics default
\paperfontsize default
\spacing single 
\papersize Default
\paperpackage a4
\use_geometry 1
\use_amsmath 0
\use_natbib 0
\use_numerical_citations 0
\paperorientation portrait
\leftmargin 4cm
\topmargin 2cm
\rightmargin 2cm
\bottommargin 2cm
\secnumdepth 3
\tocdepth 3
\paragraph_separation skip
\defskip medskip
\quotes_language swedish
\quotes_times 2
\papercolumns 1
\papersides 1
\paperpagestyle default

\layout Title

SMFS Detailed Level Design
\layout Author

Mike Pershin
\layout Date

21 February 2005
\layout Standard
\pagebreak_bottom \noindent 

\begin_inset LatexCommand \tableofcontents{}

\end_inset 


\layout Section


\noun on 
Functional Specification
\layout Subsection

SMFS operations
\layout Subsubsection

Initialization and data structures
\layout Standard

SMFS implement all needed methods as filesystem but uses other filesystem
 as store instead of block device.
 This way demands that SMFS should care about this backstore FS like VFS.
 SMFS defines several info structures for each filesystem object that contain
 all needed information about backstore related stuff and etc.
\layout Subsubsection

Plugin handling
\layout Standard

SMFS has plugin API which allows to register/deregister plugins and call
 their functions.
 Plugins are organized in linked list.
\layout Standard

Plugin initialization starts when smfs is mounting by someone with options
 that contain information about plugin needed.
 Deactivated process starts when SMFS unmount occur.
 It possible to use ioctl interface or procfs (sysfs later) for plugin managemen
t.
 Each plugin will receives notification from SMFS about filesystem operations.
 Plugins should be aware of delays, transaction handling and consistency.
\layout Standard

SMFS contains information about plugin loaded with special flags.
 There are system-wide flags and object-related to indicate exclusions.
\layout Subsubsection

Locking and plugins
\layout Standard

Every hook is called under some conditions in term of resource availability.
 SMFS should provide that all pre- and post-operation stuff should be done
 together with fs operation in backstore fs.
 Plugins should be aware about these conditions and acts properly.
\layout Subsubsection*


\emph on 
Inode_operations
\layout Standard

locking rules: all may block, none have BKL.
 Protection provided through using i_sem semaphore in inode.
\layout Standard


\begin_inset  Tabular
<lyxtabular version="3" rows="17" columns="3">
<features>
<column alignment="center" valignment="top" leftline="true" width="0">
<column alignment="center" valignment="top" leftline="true" width="0">
<column alignment="center" valignment="top" leftline="true" rightline="true" width="0">
<row topline="true" bottomline="true">
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text

\layout Standard

\end_inset 
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text

\layout Standard

i_sem
\end_inset 
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
\begin_inset Text

\layout Standard

Notes
\end_inset 
</cell>
</row>
<row topline="true">
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text

\layout Standard

lookup
\end_inset 
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text

\layout Standard

yes
\end_inset 
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
\begin_inset Text

\layout Standard

\end_inset 
</cell>
</row>
<row topline="true">
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text

\layout Standard

create
\end_inset 
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text

\layout Standard

yes
\end_inset 
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
\begin_inset Text

\layout Standard

\end_inset 
</cell>
</row>
<row topline="true">
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text

\layout Standard

link
\end_inset 
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text

\layout Standard

yes
\end_inset 
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
\begin_inset Text

\layout Standard

both
\end_inset 
</cell>
</row>
<row topline="true">
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text

\layout Standard

mknod
\end_inset 
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text

\layout Standard

yes
\end_inset 
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
\begin_inset Text

\layout Standard

\end_inset 
</cell>
</row>
<row topline="true">
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text

\layout Standard

symlink
\end_inset 
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text

\layout Standard

yes
\end_inset 
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
\begin_inset Text

\layout Standard

\end_inset 
</cell>
</row>
<row topline="true">
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text

\layout Standard

mkdir
\end_inset 
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text

\layout Standard

yes
\end_inset 
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
\begin_inset Text

\layout Standard

\end_inset 
</cell>
</row>
<row topline="true">
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text

\layout Standard

unlink
\end_inset 
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text

\layout Standard

yes
\end_inset 
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
\begin_inset Text

\layout Standard

both
\end_inset 
</cell>
</row>
<row topline="true">
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text

\layout Standard

rmdir
\end_inset 
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text

\layout Standard

yes
\end_inset 
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
\begin_inset Text

\layout Standard

both
\end_inset 
</cell>
</row>
<row topline="true">
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text

\layout Standard

rename
\end_inset 
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text

\layout Standard

yes
\end_inset 
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
\begin_inset Text

\layout Standard

all
\end_inset 
</cell>
</row>
<row topline="true">
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text

\layout Standard

truncate
\end_inset 
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text

\layout Standard

yes
\end_inset 
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
\begin_inset Text

\layout Standard

\end_inset 
</cell>
</row>
<row topline="true">
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text

\layout Standard

setattr
\end_inset 
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text

\layout Standard

yes
\end_inset 
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
\begin_inset Text

\layout Standard

\end_inset 
</cell>
</row>
<row topline="true">
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text

\layout Standard

getattr
\end_inset 
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text

\layout Standard

no
\end_inset 
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
\begin_inset Text

\layout Standard

\end_inset 
</cell>
</row>
<row topline="true">
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text

\layout Standard

setxattr
\end_inset 
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text

\layout Standard

yes
\end_inset 
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
\begin_inset Text

\layout Standard

\end_inset 
</cell>
</row>
<row topline="true">
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text

\layout Standard

getxattr
\end_inset 
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text

\layout Standard

no
\end_inset 
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
\begin_inset Text

\layout Standard

\end_inset 
</cell>
</row>
<row topline="true">
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text

\layout Standard

listxattr
\end_inset 
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text

\layout Standard

no
\end_inset 
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
\begin_inset Text

\layout Standard

\end_inset 
</cell>
</row>
<row topline="true" bottomline="true">
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text

\layout Standard

removexattr
\end_inset 
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text

\layout Standard

yes
\end_inset 
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
\begin_inset Text

\layout Standard

\end_inset 
</cell>
</row>
</lyxtabular>

\end_inset 


\layout Itemize

Additionally, ->rmdir(), ->unlink() and ->rename() have ->i_sem on victim.
\layout Itemize

cross-directory ->rename() has (per-superblock) ->s_vfs_rename_sem.
\layout Itemize

->truncate() is never called directly - it's a callback, not a method.
 It's called by vmtruncate() - library function normally used by ->setattr().
 Locking information above applies to that call (i.e.
 is inherited from ->setattr() - vmtruncate() is used when ATTR_SIZE had
 been passed).
 
\layout Standard

All inode operations may block so plugin can also block.
 All critical inode operations is already protected by semaphore i_sem.
\layout Subsubsection*


\emph on 
File operations
\layout Standard

Locking rules: file operations may block and no BKL here.
 
\layout Standard

Some hooks placed around file operations so protection needed to provide
 sync execution of pre- and post-hooks with backfs operation.
 
\layout Subsubsection*


\emph on 
Superblock operations
\layout Standard

Locking rules: All may block, only 
\emph on 
put_super 
\emph default 
is under BKL.
 
\layout Standard

No hooks here, so no actions are needed.
\layout Subsubsection*

File operations
\layout Standard

locking rules: All may block.
 
\layout Standard

None is under BKL except 
\emph on 
ioctl
\layout Comment

Taken from ..../linux/Documentation/filesystems/Locking
\layout Subsubsection

Upcall handling
\layout Standard

There is upcall API in SMFS.
 Upcalls are organized in linked list and called one by one where needed.
 
\layout Subsubsection

Fsfilt operations
\layout Standard

LVFS has fsfilt operations for SMFS.
 SMFS store copy of fsfilt operations from backstore filesystem.
 If it gets fsfilt operations then it is passed to backstore filesysem.
 
\layout Subsubsection

Transactions handling
\layout Standard

All hooks should be in the same transaction with backstore fs operation.
 To provide this SMFS starts transaction before first hook and commit after
 last one.
 SMFS care about transaction where it is needed, but plugins can add some
 stuff to it.
 To solve this SMFS pass transaction handler to plugin, so it can do transaction
 in right way.
 Also plugin should provide method to calculate extra size for transaction.
\layout Subsection

Backstore filesystem
\layout Subsubsection

Initialization
\layout Enumerate

Backstore FS is mounted by SMFS while initialization, if mount is failed
 SMFS initialization is also failed.
\layout Enumerate

SMFS creates own superblock using backstore one and fsfilter operations
\layout Subsubsection

Superblock operations
\layout Standard

SMFS have to initialize own inode structure reading inode with same number
 from backstore FS.
 Operations with such inode will be redirected to backstore FS if they are
 invoked.
 SMFS will store backstore FS inode to avoid reading inode every time.
 
\layout Subsubsection

Inode operations
\layout Standard

SMFS creates own inode operations for each inode.
 When some operations in invoked SMFS calls real filesystem method to complete
 it.
 
\layout Standard

There are additional actions for making SMFS inode and real inode consistent.
 These actions are needed before real operation to create artificial objects
 and after - to copy changes from backfs inode.
 
\layout Subsubsection

File operations
\layout Standard

SMFS creates own filp, duplicate backstore FS filp and dentry.
 They are used in file operations following by backstore operation call.
 To do file operations we create artificial file object before calling of
 real operation.
\layout Section


\noun on 
Use cases
\layout Subsection

Backstore filesystem
\layout Standard

Next options is passed to smfs and descript backstore FS completely:
\layout Standard

smfs dev=/mnt type=ldiskfs
\layout Subsection

SMFS plugins/upcalls
\layout Subsubsection

SMFS plugin activation
\layout Standard

Now all plugins are compiled in SMFS module and plugins are setted up via
 options when SMFS is mounting.
 Options should be passed as mountfsoptions like this: 
\emph on 
...
 [kml|cache|snap...]
\layout Subsubsection

SMFS plugin registration
\layout Standard

Plugins should registers with SMFS:
\layout LyX-Code

smfs_register_plugin(parameters);
\layout Standard

Parameters are struct smfs_plugin filled with valid data:
\layout Itemize

type of plugin - to distinguish it from anothers,
\layout Itemize

pre_op function - function that will be called before fs operation,
\layout Itemize

post_op function - function that will be called after fs operation,
\layout Itemize

helper function - helper function.
 See Plugin API,
\layout Itemize

any private data - data which will be return to plugin in each call.
\layout Subsubsection

SMFS hook invokation
\layout Standard

SMFS place special wrapper in own filesystem operations to call plugins
 hook:
\layout LyX-Code

/* this is wrapper that calls all hooks walking through the list */
\layout LyX-Code

SMFS_HOOK(opcode, parameters);
\layout LyX-Code

\layout Standard

Hooks are placed in SMFS methods before and after calling backstore FS operation
s:
\layout LyX-Code

/* this is how to SMFS uses hooks */
\layout LyX-Code

smfs_some_op()
\layout LyX-Code

{ 
\layout LyX-Code

        struct inode * backfs_inode = I2CI(inode);
\layout LyX-Code

        struct smfs_file_info *sfi;
\layout LyX-Code

       
\layout LyX-Code

        SMFS_HOOK(hook_opcode, ...);
\layout LyX-Code

        backfs_inode->i_fop->some_op(sfi->c_file, ...);
\layout LyX-Code

        SMFS_HOOK(hook_opcode, ...);
\layout LyX-Code

}
\layout Subsubsection

SMFS upcall usage
\layout Standard

Upcalls can be placed in any place in SMFS, where side modules wants to
 take control from SMFS.
 Module should:
\layout Enumerate

prepares function to handle upcall events,
\layout Enumerate

registers upcall using upcall API,
\layout Enumerate

receives upcalls and handles they,
\layout Enumerate

deregisters upcall when smfs is unmounted.
\layout Section


\noun on 
Logic
\noun default 
 
\noun on 
Specification
\layout Subsection

SMFS operations
\layout Subsubsection

Initialization and data structures
\layout Standard


\begin_inset ERT
status Open

\layout Standard

\backslash 
lst{../../include/linux/lustre_smfs.h}{Superblock info}{firstline=87,lastline=108}
\end_inset 


\layout LyX-Code

struct smfs_inode_info {
\layout LyX-Code

        /* this first part of struct should be 
\layout LyX-Code

           the same as in mds_info_info */
\layout LyX-Code

        struct lustre_id smi_id;
\layout LyX-Code

        
\layout LyX-Code

        /* smfs part.
 */
\layout LyX-Code

        struct inode * backfs_inode;
\layout LyX-Code

        __u32  smi_flags; //plugins pre_inode flags
\layout LyX-Code

        struct list_head plist; /* list of plugins inode info */
\layout LyX-Code

}; 
\layout Standard


\begin_inset ERT
status Open

\layout Standard

\backslash 
lst{../../include/linux/lustre_smfs.h}{File info}{firstline=113,lastline=117}
\end_inset 


\layout Subsubsection

Fsfilt operations
\layout Standard

Fsfilt operations for SMFS just call the same operations in backstore FS.
\layout Standard


\begin_inset ERT
status Open

\layout Standard

\backslash 
lst{../../lvfs/fsfilt_smfs.c}{Fsfilt redirection example}{firstline=363,lastline=383}
\end_inset 


\layout Subsubsection

Transactions handling
\layout Standard

Each operations with hooks pass transaction handler to hook.
 Hook can use it to decide does it need start new transaction or not.
\layout Standard

If there is no handler, hook can create own transaction:
\layout LyX-Code

hook_func_in_plugin()
\layout LyX-Code

{
\layout LyX-Code

        ...
\layout LyX-Code

        if (!handle) {
\layout LyX-Code

                handle = smfs_trans_start(inode, KML_CACHE_NOOP, NULL);
\layout LyX-Code

        }
\layout LyX-Code

        ...
\layout LyX-Code

}
\layout Standard

It is important that plugin should also commit transaction in the same call.
 
\layout Standard

If transaction begun before hook call it should know size of transaction,
 so we should provide a way to calculate this size through all plugins involved.
 Each plugin can register upcall with related type that will return extra
 size needed by that plugin.
 SMFS will walk through upcalls and gets total extra size for transaction:
 
\layout LyX-Code

fsfilt_smfs_start(struct inode * inode, int op, ...)
\layout LyX-Code

{
\layout LyX-Code

        ...
\layout LyX-Code

        SMFS_TRANS_EXTRA_SIZE(inode, op); /* walking through list of plugins
 */
\layout LyX-Code

        handle = cache_fsfilt->fs_start(cache_inode, op, ...);
\layout LyX-Code

        ...
\layout LyX-Code

}
\layout Comment

still have no good idea how to pass calculated size instead op without changes
 in fsfilt_ext3/any_other_fs code.
 New parameter in fs_start?
\layout Subsection

Backstore filesystem
\layout Subsubsection

Initialization/deinitialization
\layout Standard

Backstore FS is mounted by SMFS while SMFS initialization.
\layout Standard


\begin_inset ERT
status Open

\layout Standard

\backslash 
lst{../../smfs/smfs_lib.c}{Mount/Umount}{firstline=96,lastline=149}
\end_inset 


\layout Subsubsection

Inode\SpecialChar ~
info init/clear
\layout Standard

For each new inode SMFS creates private structures and getting real inode
 from backstore FS.
 SMFS is only user of backfs inode, so it is enough to get it here and put
 it down in clear_inode_info.
 Some plugins can have as SMFS-wide info as per-inode info.
 Therefore they can also participate in init_inode_info/clear_inode_info
 methods.
 Here are helpers for this.
\layout LyX-Code

static void smfs_init_inode_info(struct inode *inode, void *opaque) {
\layout LyX-Code

        struct inode *cache_inode = NULL;
\layout LyX-Code

        struct smfs_iget_args *sargs;
\layout LyX-Code

        unsigned long ino;
\layout LyX-Code

\layout LyX-Code

        sargs = (struct smfs_iget_args *)opaque;
\layout LyX-Code

        /* getting backing fs inode.
 */
\layout LyX-Code

        ino = sargs ? sargs->s_ino : inode->i_ino;
\layout LyX-Code

        cache_inode = iget(S2CSB(inode->i_sb), ino); 
\layout LyX-Code

        OBD_ALLOC(inode->u.generic_ip, sizeof(struct smfs_inode_info));
\layout LyX-Code

        I2CI(inode) = cache_inode;
\layout LyX-Code

        post_smfs_inode(inode, cache_inode);
\layout LyX-Code

        sm_set_inode_ops(cache_inode, inode);
\layout LyX-Code

        if (sargs && sargs->s_inode)
\layout LyX-Code

                I2SMI(inode)->smi_flags = I2SMI(sargs->s_inode)->smi_flags;
\layout LyX-Code

        SMFS_PLUGIN_HELPER(PL_INIT_INODE, inode);
\layout LyX-Code

}
\layout LyX-Code

\layout LyX-Code

static void smfs_clear_inode_info(struct inode *inode) {
\layout LyX-Code

        if (I2SMI(inode)) {
\layout LyX-Code

                struct inode *cache_inode = I2CI(inode);
\layout LyX-Code

                if (cache_inode != cache_inode->i_sb->s_root->d_inode)
\layout LyX-Code

                        iput(cache_inode);
\layout LyX-Code

                SMFS_PLUGIN_HELPER(PL_CLEAR_INODE, inode);
\layout LyX-Code

                OBD_FREE(inode->u.generic_ip, sizeof(struct smfs_inode_info));
\layout LyX-Code

                inode->u.generic_ip = NULL;
\layout LyX-Code

        }
\layout LyX-Code

}
\layout LyX-Code

\layout Subsubsection

Getting inode
\layout Comment

test_inode and set_inode are called under spinlock, so context-switching
 is not allowed here.
 Inode will be read after iget5_locked call 
\layout LyX-Code

int smfs_test_inode(struct inode * inode, void * opaque)
\layout LyX-Code

{
\layout LyX-Code

        struct smfs_iget_args * sargs = opaque;
\layout LyX-Code

        struct smfs_up_message message;
\layout LyX-Code

 
\layout LyX-Code

        message.inode = inode;
\layout LyX-Code

        message.param = opaque;
\layout LyX-Code

        
\layout LyX-Code

        if (sargs && (inode->i_ino == sargs->s_ino)) {
\layout LyX-Code

                /* some module can add extra checks here */
\layout LyX-Code

                if (SMFS_UPCALL(SMFS_UP_TEST_INODE, (void*)&message))
\layout LyX-Code

                        return 1;
\layout LyX-Code

        }
\layout LyX-Code

        return 0;
\layout LyX-Code

}
\layout LyX-Code

\layout LyX-Code

int smfs_set_inode(struct inode *inode, void *opaque)
\layout LyX-Code

{
\layout LyX-Code

        struct smfs_up_message message;
\layout LyX-Code

 
\layout LyX-Code

        message.inode = inode;
\layout LyX-Code

        message.param = opaque;
\layout LyX-Code

\layout LyX-Code

        /* someone can wants to do action here */
\layout LyX-Code

        SMFS_UPCALL(SMFS_UP_SET_INODE, (void*)&message);
\layout LyX-Code

        return 0; 
\layout LyX-Code

}
\layout LyX-Code

\layout LyX-Code

struct inode * smfs_iget(struct super_block * sb, ino_t hash,
\layout LyX-Code

                         struct smfs_iget_args * sargs)
\layout LyX-Code

{
\layout LyX-Code

        struct inode *inode;
\layout LyX-Code

        inode = iget5_locked(sb, hash, smfs_test_inode,
\layout LyX-Code

                             smfs_set_inode, sargs);
\layout LyX-Code

        if (inode) {
\layout LyX-Code

                if (inode->i_state & I_NEW) {
\layout LyX-Code

                        smfs_init_inode_info(inode, sargs);
\layout LyX-Code

                        unlock_new_inode(inode);
\layout LyX-Code

                }
\layout LyX-Code

                inode->i_ino = hash;
\layout LyX-Code

        }
\layout LyX-Code

        return inode;
\layout LyX-Code

}
\layout Subsubsection

Superblock operations
\layout Paragraph

read\SpecialChar ~
inode2()
\layout LyX-Code

smfs_read_inode2 (struct inode * inode, void * opaque) 
\layout LyX-Code

{
\layout LyX-Code

        smfs_init_inode_info(inode, opaque);
\layout LyX-Code

}
\layout Paragraph

dirty\SpecialChar ~
inode()/write\SpecialChar ~
inode()
\layout LyX-Code

smfs_dirty/write_inode(struct inode * inode)
\layout LyX-Code

{
\layout LyX-Code

        backfs_inode = I2CI(inode);
\layout LyX-Code

        backfs_sb->s_op->dirty/write_inode(backfs_inode);
\layout LyX-Code

        duplicate_inode(inode, backfs_inode);
\layout LyX-Code

}
\layout Paragraph

put\SpecialChar ~
inode(inode)
\layout LyX-Code

smfs_put_inode(struct inode * inode)
\layout LyX-Code

{
\layout LyX-Code

        return;
\layout LyX-Code

}
\layout Paragraph

delete\SpecialChar ~
inode()
\layout LyX-Code

smfs_delete_inode(struct inode * inode) {
\layout LyX-Code

        clear_inode(inode);
\layout LyX-Code

}
\layout Paragraph

clear\SpecialChar ~
inode()
\layout LyX-Code

smfs_clear_inode(struct inode * inode) {
\layout LyX-Code

        smfs_clear_inode_info(inode);
\layout LyX-Code

}
\layout Paragraph

put\SpecialChar ~
super(super)
\layout LyX-Code

smfs_cleanup_hooks();
\layout LyX-Code

smfs_umount_cache(smfs_super_info);
\layout LyX-Code

smfs_cleanup_smb(sb);    
\layout Paragraph

write\SpecialChar ~
super()/write\SpecialChar ~
super\SpecialChar ~
lockfs()/unlockfs()/statfs()/remountfs()
\layout LyX-Code

backfs_sb = S2CSB(sb);
\layout LyX-Code

backfs_sb->s_op->...;
\layout LyX-Code

duplicate-sb(sb, backfs_sb);
\layout Subsubsection

Inode operations
\layout Standard

Each inode operaion uses backfs inode structure, this structure is created
 while several fs operations are invoked.
 Logic of all operations is next:
\layout Itemize

There is smfs dentry for each operation passed as parameter
\layout Itemize

SMFS creates artificial dentries for using they in backstore fs operation
\layout Itemize

after operation there is backstore fs inode in backfs_dentry->d_inode
\layout Itemize

if SMFS inode not exits it is created here and connected to backfs inode
\layout Itemize

several fields are copied from backstore fs inode to smfs one
\layout Itemize

all artificial dentries are cleared.
\layout Standard


\emph on 
Artificial dentry handling
\layout LyX-Code

struct dentry *pre_smfs_dentry(struct dentry *parent_dentry, struct inode
 *cache_inode,
\layout LyX-Code

                               struct dentry *dentry) {
\layout LyX-Code

        struct dentry *cache_dentry = NULL;
\layout LyX-Code

        cache_dentry = d_alloc(parent_dentry, &dentry->d_name);
\layout LyX-Code

        if (!cache_dentry)
\layout LyX-Code

                RETURN(NULL);
\layout LyX-Code

        if (!parent_dentry)
\layout LyX-Code

                cache_dentry->d_parent = cache_dentry;
\layout LyX-Code

        if (cache_inode)
\layout LyX-Code

                d_add(cache_dentry, cache_inode);
\layout LyX-Code

        RETURN(cache_dentry);
\layout LyX-Code

}
\layout LyX-Code

\layout LyX-Code

void post_smfs_dentry(struct dentry *cache_dentry) {
\layout LyX-Code

        if (!cache_dentry)
\layout LyX-Code

                return;
\layout LyX-Code

        d_unalloc(cache_dentry);
\layout LyX-Code

} 
\layout Standard

For inode operation we have parent inode and dentry for operation.
 So SMFS has to create artificial parent dentry with backfs_inode connected
 to it and artificial dentry for ext3 operation.
 This is done with pre_smfs_dentry() method.
\layout Standard

After successfull creation SMFS will do backfs operation and gets filled
 artificial dentry.
 Next step is getting SMFS inode from cache using ext3 inode number.
 If inode is found it is connected to smfs dentry and operation can be counted
 as completed.
 SMFS clean all artificial dentries and exits.
\layout Standard


\emph on 
duplicate_inode details
\layout Standard


\begin_inset ERT
status Open

\layout Standard

\backslash 
lst{../../include/linux/lustre_smfs.h}{Duplicate
\backslash 
_inode()}{firstline=298,lastline=317}
\end_inset 


\layout Standard


\emph on 
Inode operations
\layout Standard


\begin_inset ERT
status Open

\layout Standard

\backslash 
lst{../../smfs/dir.c}{Create()}{firstline=44,lastline=577}
\end_inset 


\layout Standard


\begin_inset ERT
status Open

\layout Standard

\backslash 
lst{../../smfs/file.c}{Truncate()}{firstline=378,lastline=533}
\end_inset 


\layout Subsubsection

File operations
\layout Standard

SMFS creates artificial struct file object for each SMFS file struct and
 use it for backstore fs operations.
 Backstore struct file is created in open() method and connected to private
 field in smfs struct file.
 This struct will be released in smfs_release().
\layout Standard

When created and modified this struct file is duplicated to smfs one:
\layout Standard


\emph on 
duplicate_file
\layout Standard


\begin_inset ERT
status Open

\layout Standard

\backslash 
lst{../../include/linux/lustre_smfs.h}{duplicate
\backslash 
_file()}{firstline=343,lastline=361}
\end_inset 


\layout Paragraph

Common case (write/read/llseek/mmap/ioctl/readdir)
\layout LyX-Code

{ 
\layout LyX-Code

        struct inode * backfs_inode;
\layout LyX-Code

        struct smfs_file_info *sfi;
\layout LyX-Code

       
\layout LyX-Code

        backfs_inode = I2CI(file->f_dentry->d_inode);
\layout LyX-Code

        sfi = F2SMFI(file);
\layout LyX-Code

        pre_smfs_inode(file->f_dentry->d_inode, backfs_inode);
\layout LyX-Code

        SMFS_HOOK(..., hook_opcode , ..., PRE_HOOK, ...);
\layout LyX-Code

        if (backfs_inode->i_fop->...)
\layout LyX-Code

                backfs_inode->i_fop->...(sfi->c_file, ...);
\layout LyX-Code

        SMFS_HOOK(..., hook_opcode, ..., POST_HOOK, ...);
\layout LyX-Code

        post_smfs_inode(file->f_dentry->d_inode, backfs_inode); /* duplicate_ino
de */
\layout LyX-Code

        duplicate_file(file, sfi->c_file);
\layout LyX-Code

} 
\layout Paragraph

fsync()
\layout LyX-Code

int smfs_fsync(struct file *file, struct dentry *dentry, int datasync)
\layout LyX-Code

{
\layout LyX-Code

        struct smfs_file_info *sfi = NULL;
\layout LyX-Code

        struct dentry *backfs_dentry = NULL;
\layout LyX-Code

        struct file *backfs_file = NULL;
\layout LyX-Code

        struct inode *backfs_inode = NULL;
\layout LyX-Code

\layout LyX-Code

        backfs_inode = I2CI(dentry->d_inode);
\layout LyX-Code

        backfs_dentry = pre_smfs_dentry(NULL, backfs_inode, dentry);
\layout LyX-Code

        if (file) {
\layout LyX-Code

                sfi = F2SMFI(file);
\layout LyX-Code

                backfs_file = sfi->c_file;
\layout LyX-Code

        } 
\layout LyX-Code

        pre_smfs_inode(dentry->d_inode, backfs_inode);
\layout LyX-Code

        if (backfs_inode->i_fop->fsync) {
\layout LyX-Code

                rc = backfs_inode->i_fop->fsync(backfs_file, backfs_dentry,
 datasync);
\layout LyX-Code

        }
\layout LyX-Code

        post_smfs_inode(dentry->d_inode, backfs_inode);
\layout LyX-Code

        duplicate_file(file, backfs_file);
\layout LyX-Code

        post_smfs_dentry(backfs_dentry);
\layout LyX-Code

} 
\layout Paragraph

open()
\layout LyX-Code

int smfs_open(struct inode *inode, struct file *filp) 
\layout LyX-Code

{
\layout LyX-Code

        struct inode *backfs_inode = NULL;
\layout LyX-Code

        
\layout LyX-Code

        backfs_inode = I2CI(inode);
\layout LyX-Code

        smfs_init_cache_file(inode, filp);
\layout LyX-Code

        if (backfs_inode->i_fop->open)
\layout LyX-Code

                rc = cache_inode->i_fop->open(backfs_inode, F2CF(filp));
\layout LyX-Code

        duplicate_file(filp, F2CF(filp));
\layout LyX-Code

} 
\layout Paragraph

release()
\layout LyX-Code

int smfs_release(struct inode *inode, struct file *filp) 
\layout LyX-Code

{
\layout LyX-Code

        struct inode *backfs_inode = NULL;
\layout LyX-Code

        struct file *backfs_file = NULL;
\layout LyX-Code

        struct smfs_file_info *sfi = NULL;
\layout LyX-Code

\layout LyX-Code

        backfs_inode = I2CI(inode);
\layout LyX-Code

        if (filp) {
\layout LyX-Code

                sfi = F2SMFI(filp);
\layout LyX-Code

                backfs_file = sfi->c_file;
\layout LyX-Code

        }
\layout LyX-Code

        if (backfs_inode->i_fop->release)
\layout LyX-Code

                backfs_inode->i_fop->release(backfs_inode, backfs_file);
\layout LyX-Code

        post_smfs_inode(inode, backfs_inode);
\layout LyX-Code

        smfs_cleanup_cache_file(filp);
\layout LyX-Code

} 
\layout Section

State Specification
\layout Subsection

Transactions
\layout Standard

There are several situations are possible with transactions.
 SMFS doesn't know will plugin do transaction or not.
 
\layout Subsubsection

Backstore FS will do transaction
\layout Standard

In that case plugins actions should be in same transaction with backstore
 FS.
 Next actions will be done:
\layout Enumerate

SMFS gets extra size for current operation from all plugins.
\layout Enumerate

If extra size is not null then some plugins will participate in transaction.
 SMFS starts transaction with calculated size.
\layout Enumerate

SMFS call pre_hook.
\layout Enumerate

SMFS call backstore FS operation.
\layout Enumerate

SMFS call post_hook.
\layout Enumerate

SMFS commit transaction.
\layout Subsubsection

Backstore FS operation is not journaled
\layout Standard

In that case plugins may don't care about external conditions and do transaction
 by own:
\layout Enumerate

SMFS call pre_hook or post_hook
\layout Enumerate

Plugin starts transaction.
 
\layout Enumerate

Plugin do what it should.
\layout Enumerate

Plugin commit transaction.
\layout Standard

Transaction MUST be commited where it was started.
 It is not allowed to start it in pre_hook and commit in post_hook.
\layout Subsection

Locking
\layout Subsubsection

Inode operations
\layout Enumerate

VFS calls SMFS operation and takes i_sem.
\layout Enumerate

SMFS does pre_hook operation, calls backstore FS operation and then - post_hook.
\layout Standard

All operations are protected by i_sem in SMFS inode.
 But there are possible situations when plugin hook sequence will be like
 this:
\layout Itemize

pre_hook(inode1)
\layout Itemize

pre_hook(inode2)
\layout Itemize

post_hooks(...)
\layout Standard

This is possible because inode1 may block and operation for inode2 will
 starts.
 Plugin should be aware about that and use additional protection if it is
 needed.
 
\layout Subsubsection

File operations
\layout Standard

File operations are not protected right now.
 SMFS can also use i_sem to protect these operations, but previous words
 about operations order are make sense here also.
\the_end