Eigenfaces for Face Recognition ECE 533 Final Project Report, Fall ‘03 Min Luo, Department of Biomedical Engineering u!apat Panitc"o#, Department of Electrical $ Computer Eng ineering Abstract %Eigenface& approac" ap proac" for face recognition i& implemented a& our o ur final project' Face recognition recognition "a& #een an acti(e acti(e area of re&earc" re&earc" !it" numerou& numerou& application& application& &ince late )*+0&' Eigenface approac" i& one of t"e earlie&t appearance#a&ed face recognition met"od&, !"ic" !a& de(eloped #- M' .ur/ and ' Pentland 1)2 in )**)' ."i& met"od utilie& t"e idea of t"e principal component anal-&i& and decompo&e& face image& into a &mall &et of c"aracteri&tic feature image& called eige eigenf nfac ace& e&'' Recog Recogni niti tion on i& perf perfor orme med d #- proj projec ecti ting ng a ne! face face onto onto a lo! lo! dimen&ional linear 4face &pace defined #- t"e eigenface&, follo!ed #- computing t"e di&tance #et!een t"e re&ultant po&ition in t"e face &pace and t"o&e of /no!n face cla&&e&' num#er of e6periment& !ere done to e(aluate t"e performance of t"e face recognitio recognition n &-&tem &-&tem !e "a(e de(eloped' ."e re&ult& demon&trate demon&trate t"at t"e eigenface approac" i& 7uite ro#u&t to "ead8face orientation, #ut &en&iti(e to &cale and illumination' t t"e end of t"e report, a couple of !a-& are &ugge&ted to impro(e t"e recognition rate' ."e report i& organied a& follo!&9 t"e fir&t part pro(ide& an o(er(ie! of face recognition algorit"m&: t"e &econd part &tate& t"e t"eort"eor- of t"e eigenf eigenface& ace& approac" approac" for face recogni recognitio tion: n: Part Part ;;; ;;; focu&e focu&e&& on implementation i&&ue&, &uc" a& &-&tem &tructure de&ign, interface and u&e of eac" functional #loc/, etc': in Part ;<, a num#er of e6periment re&ult& are demon&trated for e(aluating t"e &-&tem=& performance under different circum&tance&: Finall-, a conclu&ion i& dra!n #a&ed on t"e e6periment re&ult&, and a couple of po&&i#le impro(ement& are &ugge&ted'
I. Introduction ."e face pla-& a major role in our &ocial &ocial intercour&e intercour&e in con(e-ing con(e-ing identit- and emotion' ."e "uman a#ilit- to recognie face& i& remar/a#le' >e can recognie t"ou&and& of face& learned t"roug"out our lifetime and identif- familiar face& at a glance e(en after -ear& of &eparation' ."e &/ill i& 7uite ro#u&t, de&pite large c"ange& in t"e (i&ual &timulu& due to (ie!ing condition&, e6pre&&ion, aging, and di&traction& &uc" a& gla&&e& or c"ange& in "air&t-le' Computational model& of face& "a(e #een an acti(e area of re&earc" &ince late )*+0&, for t"e- can contri#ute not onl- to t"eoretical in&ig"t& #ut al&o to practical application&, &uc" a& criminal criminal identification identification,, &ecurit&ecurit- &-&tem&, image and film proce&&ing, proce&&ing, and "uman comput computer er inte intera ract ctio ion, n, etc' etc' ?o!e( ?o!e(er er,, de(e de(elo lopi ping ng a compu computa tati tion onal al mode modell of face face recognition i& 7uite difficult, #ecau&e face& are comple6, multidimen&ional, and &u#ject to c"ange o(er time' @enerall-, t"ere are t"ree p"a&e& for face recognition, mainl- face repre&entation, face detection, and face identification' Face repre&entation i& t"e fir&t ta&/, t"at i&, "o! to model a face' ."e !a- to repre&ent a face determine& t"e &ucce&&i(e algorit"m& of detection and identification' For t"e entr-
le(el recognition At"at i&, to determine !"et"er or not t"e gi(en image repre&ent& a face, a face categor- &"ould #e c"aracteried #- generic propertie& of all face&: and for t"e &u#ordinatele(el recognition Ain ot"er !ord&, !"ic" face cla&& t"e ne! face #elong& to, detailed feature& of e-e&, no&e, and mout" "a(e to #e a&&igned to eac" indi(idual face' ."ere are a (ariet- of approac"e& for face repre&entation, !"ic" can #e roug"l- cla&&ified into t"ree categorie&9 template#a&ed, feature#a&ed, and appearance#a&ed' ."e ."e &im &imple& ple&tt template-matching appro approac ac"e "e&& repr repre& e&en entt a !"ol !"olee face face u&in u&ing g a &ing &ingle le template, i'e', a D arra- of inten&it-, !"ic" i& u&uall- an edge map of t"e original face image' ;n a more comple6 !a- of templatematc"ing, multiple template& ma- #e u&ed for eac" face face to account account for recogni recognitio tion n from from differ different ent (ie!poi (ie!point& nt&'' not"e not"err import important ant (ariation i& to emplo- a &et of &maller facial feature template& t"at corre&pond to e-e&, no&e, and mout", for a &ingle (ie!point' ."e mo&t attracti(e ad(antage of template matc"i matc"ing ng i& t"e &impli &implicit cit-, -, "o!e(er "o!e(er,, it &uffer &uffer&& from from large large memormemor- re7uir re7uireme ement nt and inefficient inefficient matc"ing' matc"ing' ;n feature-based approac"e&, approac"e&, geometric feature&, &uc" a& po&ition and !idt" of e-e&, no&e, and mout", e-e#ro!& t"ic/ne&& and arc"e&, face #readt", or in(ariant moment&, are e6tracted to repre&ent a face' Feature#a&ed approac"e& "a(e &maller memor- re7uirement and a "ig"er recognition &peed t"an template#a&ed one& do' ."e- are particularl- u&eful for face &cale normaliation and 3D "ead model#a&ed po&e e&ti e&tima mati tion on'' ?o!e ?o!e(e (er, r, perfe perfect ct e6tr e6trac acti tion on of feat featur ure& e& i& &"o!n &"o!n to #e diff diffic icul ultt in implementation 152' ."e idea of appearance-based approac"e& approac"e& i& to project face image& onto onto a linear linear &u#&pa &u#&pace ce of lo! dimen&i dimen&ion& on&'' uc" uc" a &u#&pa &u#&pace ce i& fir&t fir&t con&truct con&tructed ed # principal component anal-&i& on a &et of training image&, !it" eigenface& a& it& eigen(ector&' Later, t"e concept of eigenface& !ere e6tended to eigenfeature&, &uc" a& eigeneeigene-e&, e&, eigenm eigenmout out", ", etc' etc' for t"e detect detection ion of facial facial featur feature& e& 12' 12' More More recent recentll-,, fi&"erface &pace 1G2 and illumination &u#&pace 1+2 "a(e #een propo&ed for dealing !it" recognition under (ar-ing illumination' Face detection detection i& to locate locate a face in a gi(en image image and to &eparate &eparate it from t"e remaining remaining &cene' e(eral approac"e& "a(e #een propo&ed to fulfil t"e ta&/' Hne of t"em i& to utilie t"e elliptical &tructure of "uman "ead 1*2' ."i& met"od locate& t"e "ead outline #- t"e Cann-& edge finder and t"en fit& an ellip&e to mar/ t"e #oundar- #et!een t"e "ead region and t"e #ac/ground' ?o!e(er, t"i& met"od i& applica#le onl- to frontal (ie!&, t"e detection of nonfrontal (ie!& need& to #e in(e&tigated' &econd approac" for face detection manipulate& t"e image& in 4face &pace 1)2' ;mage& of face& do not c"ange radicall- !"en projected into t"e face &pace, !"ile projection& of nonface image& appear 7uite different' ."i& #a&ic idea i& uded to detect t"e pre&ence of face& in a &cene9 at e(erlocation location in t"e image, calculate calculate t"e di&tance di&tance #et!een #et!een t"e local &u#image and face &pace' ."i& di&tance from face &pace i& u&ed a& a mea&ure of 4facene&&, &o t"e re&ult of calculating t"e di&tance from face &pace at e(er- point in t"e image i& a 4face map' Lo! (alue&, in ot"er !ord&, &"ort di&tance& from face &pace, in t"e face map indicate t"e pre&ence of a face' Face identification i& performed at t"e &u#ordinatele(el' t t"i& &tage, a ne! face i& compared to face model& &tored in a data#a&e and t"en cla&&ified to a /no!n indi(idual if a corre&ponden corre&pondence ce i& found' ."e performance performance of face identificati identification on i& affected #- &e(eral &e(eral factor&9 &cale, po&e, illumination, facial e6pre&&ion, and di&gui&e' ."e scale ."e scale of of a face can #e "andled #- a re&caling proce&&' ;n eigenface approac", t"e &cal &calin ing g fact factor or can can #e dete determ rmin ined ed #- mult multip iple le tria trial& l&'' ."e ."e idea idea i& to u&e u&e mult multi& i&cal calee
eigenface&, in !"ic" a te&t face image i& compared !it" eigenface& at a num#er of &cale&' ;n t"i& ca&e, t"e image !ill appear to #e near face &pace of onl- t"e clo&e&t &caled eigenface&' E7ui(alentl-, !e can &cale t"e te&t image to multiple &ie& and u&e t"e &caling factor t"at re&ult& in t"e &malle&t di&tance to face &pace'
II. Eigenfaces for Recognition Before t"e pu#lication of 1)2, muc" of t"e !or/ on automated face recognition "a& ignored t"e i&&ue of !"at a&pect& of t"e face &timulu& are important for identification, a&&uming t"at predefined mea&urement& !ere rele(ant and &ufficient' ;n earl- )**0&, M' .ur/ and ' Pentland "a(e realied t"at an information t"eor- approac" of coding and decoding face image& ma- gi(e in&ig"t into t"e information content of face image&, emp"a&iing t"e &ignificant local and glo#al 4feature&' uc" feature& ma- or ma- not #e directl- related to our intuiti(e notion of face feature& &uc" a& t"e e-e&, no&e, lip&, and "air' ;n t"e language of information t"eor-, t"e o#jecti(e i& to e6tract t"e rele(ant information in a face image, encode it a& efficientl- a& po&&i#le, and compare one face encoding !it" a data#a&e of model& encoded in t"e &ame !a-' &imple approac" to e6tract t"e information contained in a face image i& to &ome"o! capture t"e (ariation in a collection of face image&, independent of an- judgement of feature&, and u&e t"i& information to encode and compare indi(idual face image&' ;n mat"ematical term&, t"e o#jecti(e i& to find t"e principal component& of t"e di&tri#ution of face&, or t"e eigen(ector& of t"e co(ariance matri6 of t"e &et of face iamge&' ."e&e eigen(ector& can #e t"oug"t of a& a &et of feature& !"ic" toget"er c"aracterie t"e (ariation #et!een face image&' Eac" image location contri#ute& more or le&& to eac" eigen(ector, &o t"at !e can di&pla- t"e eigen(ector a& a &ort of g"o&tl- face called an eigenface' ome of t"e&e face& are &"o!n in Figure I' Eac" face image in t"e training &et can #e repre&ented e6actl- in term& of a linear com#ination of t"e eigenface&' ."e num#er of po&&i#le eigenface& i& e7ual to t"e num#er
of face image& in t"e training &et' ?o!e(er, t"e face& can al&o #e appro6imated u&ing onl- t"e 4#e&t eigenface&%t"o&e t"at "a(e t"e large&t eigen(alue&, and !"ic" t"erefore account for t"e mo&t (ariance !it"in t"e &et of face image&' ."e primar- rea&on for u&ing fe!er eigenface& i& computational efficienc-' ."e mo&t meaningful M eigenface& &pan an M dimen&ional &u#&pace%4face &pace%of all po&&i#le image&' ."e eigenface& are e&&entiall- t"e #a&i& (ector& of t"e eigenface decompo&ition' ."e idea of u&ing eigenface& !a& moti(ated #- a tec"ni7ue for efficientl- repre&enting picture& of face& u&ing principal component anal-&i&' ;t i& argued t"at a collection of face image& can #e appro6imatel- recon&tructed #- &toring a &mall collection of !eig"t& for eac" face and a &mall &et of &tandard picture&' ."erefore, if a multitude of face image& can #e recon&tructed #- !eig"ted &um of a &mall collection of c"aracteri&tic image&, t"en an efficient !a- to learn and recognie face& mig"t #e to #uild t"e c"aracteri&tic feature& from /no!n face image& and to recognie particular face& #- comparing t"e feature !eig"t& needed to Aappro6imatel- recon&truct t"em !it" t"e !eig"t& a&&ociated !it" t"e /no!n indi(idual&' ."e eigenface& approac" for face recognition in(ol(e& t"e follo!ing initialiation operation&9 )' c7uire a &et of training image&' ' Calculate t"e eigenface& from t"e training &et, /eeping onl- t"e #e&t M image& !it" t"e "ig"e&t eigen(alue&' ."e&e M image& define t"e 4face &pace' & ne! face& are e6perienced, t"e eigenface& can #e updated' 3' Calculate t"e corre&ponding di&tri#ution in M dimen&ional !eig"t &pace for eac" /no!n indi(idual Atraining image, #- projecting t"eir face image& onto t"e face &pace' ?a(ing initialied t"e &-&tem, t"e follo!ing &tep& are u&ed to recognie ne! face image&9 )' @i(en an image to #e recognied, calculate a &et of !eig"t& of t"e M eigenface& #- projecting t"e it onto eac" of t"e eigenface&' ' Determine if t"e image i& a face at all #- c"ec/ing to &ee if t"e image i& &ufficientl- clo&e to t"e face &pace' 3' ;f it i& a face, cla&&if- t"e !eig"t pattern a& eig"er a /no!n per&on or a& un/no!n' I' AHptional Jpdate t"e eigenface& and8or !eig"t pattern&' 5' AHptional Calculate t"e c"aracteri&tic !eig"t pattern of t"e ne! face image, and incorporate into t"e /no!n face&' Calculating Eigenfaces Let a face image Γ A6,- #e a t!odimen&ional N #- N arra- of inten&it- (alue&' n image ma- al&o #e con&idered a& a (ector of dimen&ion N C , &o t"at a t-pical image of &ie 5 #- 5 #ecome& a (ector of dimen&ion 5,53, or e7ui(alentl-, a point in 5,53 dimen&ional &pace' n en&em#le of image&, t"en, map& to a collection of point& in t"i& "uge &pace' ;mage& of face&, #eing &imilar in o(erall configuration, !ill not #e randoml- di&tri#uted in t"i& "uge image &pace and t"u& can #e de&cri#ed #- a relati(el- lo! dimen&ional &u#&pace' ."e main idea of t"e principal component anal-&i& i& to find t"e (ector t"at #e&t account for t"e di&tri#ution of face image& !it"in t"e entire image &pace' ."e&e (ector& define t"e &u#&pace of face image&, !"ic" !e call 4face &pace' Eac" (ector i& of lengt" N C , de&cri#e& an N #- N image, and i& a linear com#ination of t"e original face image&'
Becau&e t"e&e (ector& are t"e eigen(ector& of t"e co(ariance matri6 corre&ponding to t"e original face image&, and #ecau&e t"e- are faceli/e in appearance, t"e- are referred to a& 4eigenface&' Γ , K, Γ Let t"e training &et of face image& #e ) , Γ C , M ' ."e a(erage face of t"e ) M &et if defined #- Ψ = ∑Γ n ' Eac" face differ& from t"e a(erage #- t"e (ector M n =) 3
Φn = Γ n − Ψ ' n e6ample training &et i& &"o!n in Figure )a, !it" t"e a(erage face Ψ
&"o!n in Figure )#' ."i& &et of (er- large (ector& i& t"en &u#ject to principal component anal-&i&, !"ic" &ee/& a &et of M ort"onormal (ector&, µ n , !"ic" #e&t de&cri#e& t"e di&tri#ution of t"e data' ."e k t" (ector, µ k i& c"o&en &uc" t"at ) M T C λ k = A µ k Φ n B A) ∑ M n =) i& a ma6imum, &u#ject to
), l = k µ l µ k = 0, otherwise T
."e (ector& µ k and &calar& co(ariance matri6
λ k are
C =
)
A
t"e eigen(ector& and eigen(alue&, re&pecti(el-, of t"e M
∑Φ M
T
n
T
Φ n = AA
A3
n =)
!"ere t"e matri6 A = 1Φ) ΦC '''ΦM 2 ' ."e matri6 C , "o!e(er, i& N C #- N C , and determining t"e N C eigen(ector& and eigen(alue& i& an intracta#le ta&/ for t-pical image &ie&' computationall- fea&i#le met"od i& needed to find t"e&e eigen(ector&' ;f t"e num#er of data point& in t"e image &pace i& le&& t"an t"e dimen&ion of t"e &pace A M < N C , t"ere !ill #e onl- M − ) , rat"er t"an N C , meaningful eigen(ector& At"e remaining eigen(ector& !ill "a(e a&&ociated eigen(alue& of ero' Fortunatel-, !e can &ol(e for t"e N C dimen&ional eigen(ector& in t"i& ca&e #- fir&t &ol(ing for t"e eigen(ector& of and M #- M matri6%e'g', &ol(ing a ) 6 ) matri6 rat"er t"an a ),3+I 6 ),3+I matri6%and t"en ta/ing appropriate linear com#ination& of t"e face image& Φn ' Con&ider t"e eigen(ector& ν n of A T A &uc" t"at AT Aν n = λ nν n AI Premultipl-ing #ot" &ide& #- A, !e "a(e AAT Aν n = λ n Aν n
A5
are t"e eigen(ector& of C = AAT ' T Follo!ing t"i& anal-&i&, !e con&truct t"e M #- M matri6 L = AT A , !"ere Lmn = Φ m Φn , from !"ic" !e &ee t"at
Aν n
and find t"e M eigen(ector& ν n of L' ."e&e (ector& determine linear com#ination& of t"e M training &et face image& to form t"e eigenface& µ n 9 M
µ n = ∑ν nk Φ k = Aν n , n = ),'''''', M k =)
A
>it" t"i& anal-&i& t"e calculation& are greatl- reduced, from t"e order of t"e num#er of pi6el& in t"e image& A N C to t"e order of t"e num#er of image& in t"e training &et A M ' ;n practice, t"e training &et of face image& !ill #e relati(el- &mall A M < N C , and t"e calculation& #ecome 7uite managea#le' ."e a&&ociated eigen(alue& allo! u& to ran/ t"e eigen(ector& according to t"eir u&efulne&& in c"aracterieing t"e (ariation among t"e image&' Using Eigenfaces to Classify a Face Image ."e eigenface image& calculated from t"e eigen(ector& of L &pan a #a&i& &et !it" !"ic" to de&cri#e face image&' & mentioned #efore, t"e u&efulne&& of eigen(ector& (arie& according t"eir a&&ociated eigen(alue&' ."i& &ugge&t& !e pic/ up onl- t"e mo&t meaningful eigen(ector& and ignore t"e re&t, in ot"er !ord&, t"e num#er of #a&i& function& i& furt"er reduced from M to M’ A M’M and t"e computation i& reduced a& a con&e7uence' E6periment& "a(e &"o!n t"at t"e RM pi6el#-pi6el error& in repre&enting cropped (er&ion& of face image& are a#out !it" M ))5 and M’ I0 1))2' ;n practice, a &maller M’ i& &ufficient for identification, &ince accurate recon&truction of t"e image i& not a re7uirement' ;n t"i& frame!or/, identification #ecome& a pattern recognition ta&/' ."e eigenface& &pan an M’ dimen&ional &u#&pace of t"e original N C image &pace' ."e M’ mo&t &ignificant eigen(ector& of t"e L matri6 are c"o&en a& t"o&e !it" t"e large&t a&&ociated eigen(alue&' ne! face image Γ i& tran&formed into it& eigenface component& Aprojected onto 4face &pace #- a &imple operation ω n = µ n AΓ − ΨB AG for n),KK, M’ ' ."i& de&cri#e& a &et of point#-point image maltiplication& and &ummation&' T ."e !eig"t& form a (ector Ω = 1ω ) , ω C ,''', ω M D 2 t"at de&cri#e& t"e contri#ution of eac" eigenface in repre&enting t"e input face image, treating t"e eigenface& a& a #a&i& &et for face image&' ."e (ector ma- t"en #e u&ed in a &tandard pattern recognition algorit"m to find !"ic" of a num#er of predefined face cla&&e&, if an-, #e&t de&cri#e& t"e face' ."e &imple&t met"od for determining !"ic" face cla&& pro(ide& t"e #e&t de&cription of an input face image i& to find t"e face cla&& k t"at minimie& t"e Euclidian di&tance C C ε = A Ω − Ω B A+ k
k
!"ere Ωk i& a (ector de&cri#ing t"e k t" face cla&&' ."e face cla&&e& Ωk are calculated #- a(eraging t"e re&ult& of t"e eigenface repre&entation o(er a &mall num#er of face image& Aa& fe! a& one of eac" indi(idual' face i& cla&&ified a& 4un/no!n, and optionall- u&ed to created a ne! face cla&&' Becau&e creating t"e (ector of !eig"t& i& e7ui(alent to projecting t"e original face image onto to lo!dimen&ional face &pace, man- image& Amo&t of t"em loo/ing not"ing li/e a face !ill project onto a gi(en pattern (ector' ."i& i& not a pro#lem for t"e &-&tem, "o!e(er, &ince t"e di&tance ε #et!een t"e image and t"e face &pace i& &impl- t"e M D
&7uared di&tance #et!een t"e meanadju&ted input image Φ = Γ − Ψ and Φ f = ∑ ω i µ i , i =)
it& projection onto face &pace9 C
ε
= Φ−Φ f
C
A*
."u& t"ere are four po&&i#ilitie& for an input image and it& pattern (ector9 A) near face &pace and near a face cla&&: A near face &pace #ut not near a /no!n face cla&&: A3 di&tant from face &pace and near a face cla&&: AI di&tant from face &pace and not near a /no!n face cla&&' ;n t"e fir&t ca&e, an indi(idual i& recognied and identified' ;n t"e &econd ca&e, an un/no!n indi(idual i& pre&ent' ."e la&t t!o ca&e& indicate t"at t"e image i& not a face image' Ca&e t"ree t-picall- &"o!& up a& a fal&e po&iti(e in mo&t recognition &-&tem&: in t"i& frame!or/, "o!e(er, t"e fal&e recognition ma- #e detected #ecau&e of t"e &ignificant di&tance #et!een t"e image and t"e &u#&pace of e6pected face image&' Summary of Eigenface Recognition Procedure ."e eigenface& approac" for face recognition i& &ummaried a& follo!&9 )' Collect a &et of c"aracteri&tic face image& of t"e /no!n indi(idual&' ."i& &et &"ould include a num#er of image& for eac" per&on, !it" &ome (ariation in e6pre&&ion and in t"e lig"ting A&a- four image& of ten people, &o M!I0' ' Calculate t"e AI0 6 I0 matri6 L, find it& eigen(ector& and eigen(alue&, and c"oo&e t"e M’ eigen(ector& !it" t"e "ig"e&t a&&ociated eigen(alue& Alet M’ )0 in t"i& e6ample' 3' Com#ine t"e normalied training &et of image& according to E7' A to produce t"e A M’ )0 eigenface& µ k , k =),'''''', M D ' I' For eac" /no!n indi(idual, calculate t"e cla&& (ector Ωk #- a(eraging t"e eigenface pattern (ector& Ω 1from E7' A+2 calculated from t"e original Afour image& of t"e indi(idual' C"oo&e a t"re&"old θ ε t"at define& t"e ma6imum allo!a#le di&tance from an- face cla&&, and a t"re&"old t"at define& t"e ma6imum allo!a#le di&tance from face &pace 1according to E7' A*2' 5' For eac" ne! face image to #e identified, calculate it& pattern (ector Ω , t"e di&tance ε k to eac" /no!n cla&&, and t"e di&tance ε to face &pace' ;f t"e minimum di&tance ε k < θ ε and t"e di&tance ε < θ , cla&&if- t"e input face a& t"e indi(idual a&&ociated !it" cla&& (ector Ωk ' ;f t"e minimum di&tance ε k > θ ε #ut ε < θ , t"en t"e image ma- #e cla&&ified a& 4un/no!n, and optionall- u&ed to #egin a ne! face cla&&' ' ;f t"e ne! image i& cla&&ified a& a /no!n indi(idual, t"i& image ma- #e added to t"e original &et of familiar face image&, and t"e eigenface& ma- #e recalculated A&tep& )I' ."i& gi(e& t"e opportunit- to modif- t"e face &pace a& t"e &-&tem encounter& more in&tance& of /no!n face&'
III. Implementation Issues ."e entire program con&i&t& of four functional #loc/&, namel- ‘Load;mage&=, ‘Con&tructEigenface&=, ‘Cla&&if-Ne!face=, and ‘undoJpdateEigenface&=' ."ere i& al&o a ‘main= function, !"ic" call& ‘Con&tructEigenface&= and ‘Cla&&if-Ne!face= function& to complete t"e face recognition ta&/' System Structure ."e &tructure of t"e &-&tem i& &"o!n in Figure )' ;n t"e figure, t"e &7uare &"ape indicate& function&, and t"e parallogram repre&ent& file&' n arro! pointing out from a file to a function mean& t"e function load& t"e file: an arro! pointing in t"e ot"er direction
indicate& t"at t"e function create& or update& t"e file: a #idirectional arro! mean& t"e file i& fir&t read #- t"e function, and later modified or updated #- it' ."e&e file& "elp t"e ‘Con&tructEigenface&= and ‘Cla&&if-Ne!face= function& communicate !it" eac" ot"er in a !ell organied !a-' Load;mage&
Con&tructEigenface&
trainingimage&'ma t
eigenface&'mat
Cla&&if-Ne!face
facecla&&e&'mat undoJpdateEigenface& noteOeigenface&'mat Figure 1. -&tem Flo!c"art . ."e &7uare& and parallogram& repre&ent function& and file& re&pecti(el-' n arro! pointing out from a file to a function mean& t"e function read&8load& t"e file: an arro! pointing in t"e ot"er direction indicate& t"at t"e function create&8update& t"e file: a #idirectional arro! mean& t"e file i& fir&t read #- t"e function, and later modified8updated #- it' ."e&e file& "elp t"e ‘Con&tructEigenface&= and ‘Cla&&if-Ne!face= function& communicate !it" eac" ot"er in a !ell organied !a-'
Functional Blocks Eac" functional #loc/ "a& a corre&ponding 'm file "refer to the source code#' Detailed de&cription of t"e functional #loc/& i& a& follo!&' Load;mage&Aimagefilename9 Functionalit-9 load all training image& and return t"eir content& Ainten&it- (alue& • ;nput parameter&9 imagefilename%a &tring t"at &tate& an image file name' • Hutput parameter&9 ;%a 3D matri6 !"o&e component& are t"e inten&it- (alue& of • t"e training image&' J&e9 ;Load;mage&Aimagefilename • P&eudocode9 • if imagefilename is an empty string do (1 read all default training images into !" matri# I
($ sa%e I to file &trainingimages.mat' in &.)rainingSet' directory Anote9 relati(e pat" i& u&ed t"roug"out t"e document, ‘relati(e= in t"e &en&e t"at relati(e to t"e location of t"e 'm &ource file& (! *rite t+e file names of t+e default training images to te#t file ¬e,eigenfaces.t#t' in &.Eigenfaces' directory else Aa&&uming t"e file pat" and name are correct, i'e' t"e image file can #e opened and read properl- do (1 copy t+e image named imagefilename into &.)rainingSet' directory ($ load I from file &trainingimages.mat' in &.)rainingSet' directory A3 read t+e image file named imagefilename At"e input parameter and e#tract its illuminant component (i.e. intensity into $" matri# Ine* (- concatenate Ine* to I and sa%e t+e modified I to &trainingimages.mat' t+us file &trainingimages.mat' gets updated A5 append imagefilename Ai'e', name of t"e ne! training image to t+e end of te#t file ¬e,eigenfaces.t#t' in &.Eigenfaces' directory (/ copy t+e test image from &.)estImage' directory to &.)rainingSet' direcotry • Note9 ‘Load;mage&= function i& al!a-& called in t"e ‘Con&tructEigenface&= function' ."e input parameter of t"e latter i& pa&&ed to t"e former a& it& iput' ."e 3D matri6 ; returned #- ‘Load;mage&= !ill #e u&ed for furt"er computation in ‘Con&tructEigenface&='
Con&tructEigenface&Aimagefilename9 • Functionalit-9 A) con&truct or update eigenface&: A con&truct or update face cla&&e&' ;nput parameter&9 imagefilename%a &tring t"at &tate& an image file name • Hutput parameter&9 &f%indicator of &ucce&&8failure of t"e e6ecution of t"e • function' ;f &f e7ual& to ), e6ecution &ucce&&full-: if &f e7ual& to 0, e6ecution failure J&e9 &fCon&tructEigenface&Aimagefilename • P&eudocode9 • 0 call function &oadImages' and get t+e illuminant components I of current training images 0 construct eigenfaces 2 and face classes 34E56 7ased on I sa%e 2 to file &eigenfaces.mat' in &.Eigenfaces' directory sa%e 34E56 to file &faceclasses.mat' in &.Eigenfaces' directory • Note9 t"e input parameter of function ‘Con&tructEigenface&= i& pa&&ed to function ‘Load;mage&= a& it& input !"en t"e former call& t"e latter' ."erefore, t"e t!o &tatement& mar/ed !it" in a#o(e p&eudocode can #e re&tated a& follo!ing9 if imagefilename is an empty string do (1 construct t+e eigenfaces 2 7ased on t+e default training images ($ construct t+e face classes 34E56 7ased on t+e default training images
else Aa&&uming file pat" and name are correct, i'e' t"e image file can #e opened and read properl- do (1 update current 2 according to t+e ne*ly added training image ($ update current 34E56 according to t+e latest added training image >"en t"e input parameter i& an empt- &tring, ‘Con&tructEigenface&= function con&truct& t"e (er- fir&t (er&ion of eigenface&, and face cla&&e& #a&ed on t"e default training image &et' Calling t"i& function !it" an empt- &tring a& t"e input parameter i& a good t"ing to do onl- if it i& t"e fir&t time !e run t"e face recognition program: ot"er!i&e, !e ma- lo&e u&eful information' &&uming !e "a(e alread- run t"e face recognition program &e(eral time&, encountered a num#er of ne! face&, and added t"e ne! face& to our training &et, if ‘Con&tructEigenface&= function i& called !it" an empt- input &tring, file& &uc" a& ‘trainingimage&'mat=, ‘noteOeigenface&'t6t=, ‘eigenface&'mat= and ‘facecla&&e&'mat= !ill all go #ac/ to t"eir initial (er&ion, in ot"er !ord&, updated eigenface& and face cla&&e& #a&ed on t"e ne! training image& !ill #e mi&&ing and !"at !e "a(e are t"o&e containing onl- t"e default training image&= information' ."erefore, #e cautiou& !"en calling ‘Con&tructEigenface&= function !it" an empt- &tring a& t"e input parameter'
Cla&&if-Ne!faceAimagefilename9 Functionalit-9 gi(en a te&t image, t"i& function i& a#le to determine • A) >"et"er it i& a face image A ;f it i& a face image, doe& it #elong to an- of t"e e6i&ting face cla&&e&Q a' ;f &o, !"ic" face cla&& doe& it corre&pond toQ #' ;f not, Aoptionall- update t"e eigenface& according to t"e te&t image ;nput parameter9 imagefilename%a &tring t"at &tate& t"e name of t"e te&t image • Hutput parameter9 re&ult%indicator of t"e te&t image=& &tatu& • a' re&ult0, #ad file Acannot open t"e file #' re&ult, te&t image i& not a face image c' re&ult), te&t image i& a face image, and #elong& to one of t"e e6i&ting face cla&&e& d' re&ult), te&t image i& a face image, #ut doe& not #elong to an- of t"e e6i&ting face cla&&e& J&e9 re&ultCla&&if-Ne!faceAimagefilename • • P&eudocode9 if t+e test image file cannot 7e opened do (1 result89 ($ return (end if predefine t*o t+res+olding %alues θ and θ ε
pro:ect t+e test image Γ onto face space compute t+e distance ε 7et*een t+e test image and its pro:ection onto t+e face space if ε ; θ A Γ i& not a face image do (1 result8<$ ($ return (end if compute t+e distance ε k , k = ),'''''', M 7et*een Γ and eac+ face class
find minε k , k
= ),'''''', M S = ε k
if ε k = θ ε A Γ #elong& to k’ t" face cla&& do (1 display t+e test image and its corresponding training image ($ result81 else A Γ doe& not #elong to an- of t"e e6i&ting face cla&&e& do (1 (optionally call function ConstructEigenfaces *it+ t+e file name of t+e test image as t+e input parameter ($ result8<1
undoJpdateEigenface&9 Functionalit-9 A) remo(e t"e late&t added training image from t"e training &et, • follo!ed #- computation of eigenface& and face cla&&e& A o(er!rite all t"e four file& &"o!n in Figure ), mainl‘trainingimage&'mat= in ‘'T.raininget= director-, ‘eigenface&'mat=, ‘facecla&&e&'mat=, ‘noteOeigenface&'t6t= in ‘'TEigenface&= director• Note9 function ‘undoJpdateEigenface&= i& nece&&ar- for undo an update of eigenface& and face cla&&e&, !"ic" &"ould not "a(e #een done J&e9 undoJpdateEigenface& • P&eudocode • 1. load &trainingimages.mat' from &)rainingSet' directory and get !" matri# I $. remo%e t+e last slice from I (7y statement I8I(>>1>(si?e(I!<1 if programming *it+ 4ata7 !. sa%e modified I to &trainingimages.mat' in &)rainingSet' directory -. remo%e t+e last line (i.e. t+e name of t+e image file to 7e remo%ed from t+e training set from ¬e,eigenfaces.t#t' @. delete t+e lastest added training image from &)rainingSet' directory /. construct face space eigenfaces and face classes 7ased on modified I A. o%er*rite follo*ing files> &eigenfaces.mat' and &faceclasses.mat' main9 •
Functionalit-9 group function& &uc" a& ‘Con&tructEigenface&= ‘Cla&&if-Ne!face= toget"er and form t"e face recognition &-&tem
and
•
Flo!9 t"e &-&tem flo! i& illu&trated in Figure '
Con&truct8update eigenface& prior to face identificationQ N
.e&t image name
Cla&&if- t"e te&t image Figure $. System Flo*
Con&truct8update eigenface&
I2. E#periment Results ."e face recognition &-&tem !a& te&ted u&ing a &et of face image& do!nloaded from M;. Media La# &er(er1))2' ll t"e training and te&ting image& are gra-&cale image& of &ie )06)+' ."ere are ) per&on& in t"e face image data#a&e, eac" "a(ing G di&tinct picture& ta/en under different condition& Ailluminance, "ead tilt, and "ead &cale' ."e training image& are c"o&en to #e t"o&e of full "ead &cale, !it" "eadon lig"ting, and uprig"t "ead tilt' ."e initial training &et con&i&t& of ) face image& of ) indi(idual&, i'e' one image for one indi(idual A M )' ."e&e training image& are &"o!n in Figure 3a' Figure 3# i& t"e a(erage image of t"e training &et'
Figure !a. Initial )raining Images
Figure !7. 6%erage Face of Initial )raining Set
fter principal component anal-&i&, M’ )) eigenface& are con&tructed #a&ed on t"e M ) training image&' ."e eigenface& are demon&trated in Figure I' ."e a&&ociated eigen(alue& of t"e&e eigenface& are ))*'), )35'0, )G3'*, )*G'3, 30'3, 33', IG*'+, 550'0, G'+, +I3'5, )+)', in order' ."e eigen(alue& determine t"e 4u&efulne&& of t"e&e eigenface&' >e /eep all t"e eigenface& !it" nonero eigen(alue& in our e6periment,
&ince t"e &ie of our training &et i& &mall' ?o!e(er, to ma/e t"e &-&tem c"eaper and 7uic/er, !e can ignore t"o&e eigenface& !it" relati(el- &mall eigen(alue&'
Figure -. Eigenfaces
."e performance of t"e eigenface& approac" under different condition& i& &tudied a& follo!&' Recognition !it" different "ead tilt&9 ."e ro#u&tne&& of t"e eigenface& recognition algorit"m to "ead tilt i& &tudied #- te&ting face image& of eac" per&on t"at i& in t"e training &et, !it" different "ead tilt&%eit"er left oriented or rig"toriented, a& &"o!n in Figure 5'
a. 7. c. Figure @. )raining image and test images *it+ different +ead tilts. a. training image 7. test image 1 c. test image $
;f t"e &-&tem correctl- relate& t"e te&t image !it" it& corre&pondence in t"e training &et, !e &a- it conduct& a true-positi$e identification AFigure& and G: if t"e &-&tem relate& t"e te&t image !it" a !rong per&on AFigure +, or if t"e te&t image i& from an un/no!n indi(idual !"ile t"e &-&tem recognie& it a& one of t"e per&on& in t"e data#a&e, a false positi$e identifaction i& performed: if t"e &-&tem identifie& t"e te&t image a& un/no!n
!"ile t"ere doe& e6i&t a corre&pondence #et!een t"e te&t image and one of t"e training image&, t"e &-&tem conduct& a false-negati$e detection' ."e e6periment re&ult& are illu&trated in t"e .a#le )9 )a7le 1> Recognition *it+ different +ead tilts
Num#er of te&t image& Num#er of truepo&iti(e identification& Num#er of fal&epo&iti(e identification& Num#er of fal&enegati(e identification&
a.
I )) )3 0
7. c. Figure / (irfan). Recognition *it+ different +ead tiltssuccessD a. test image 1 7. test image $ c. training image
a. 7. Figure A (david). Recognition *it+ different +ead tiltssuccessD a. test image 7. training image
a.
7.
c. d. Figure (foof). Recognition *it+ different +ead tiltsfalseD a. test image 1 7. training image (irfan returned 7y t+e face recognition system c. test image $ d. training image (stephen returned 7y t+e program
Recognition !it" (ar-ing illuminance9 Eac" training image A!it" "eadon lig"ting "a& t!o corre&ponding te&t image&%one !it" lig"t mo(ed #- I5 degree& and t"e ot"er !it" lig"t mo(ed #- *0 degree&' Ht"er condition&, &uc" a& "ead &cale and tilt, remain t"e &ame a& in t"e training image' ."e e6periment re&ult& are &"o!n in .a#le ' )a7le $> Recognition *it+ %arying illuminance
Num#er of te&t image& Num#er of truepo&iti(e identification& Num#er of fal&epo&iti(e identification& Num#er of fal&enegati(e identification&
I ) 3 0
Figure * &"o!& t"e difference #et!een t"e training image and te&t image&'
Figure . )raining image and test images *it+ %arying illuminance. a. training image 7. test image 1> lig+t mo%ed 7y -@ degrees c. test image $> lig+t mo%ed 7y 9 degrees
truepo&iti(e e6ample i& demon&trated in Figure )0, and a fal&enegati(e one i& &"o!n in Figure ))'
a. 7. Figure 19 (stan). Recognition *it+ %arying illuminancesuccessD a. test image lig+t mo%ed 7y -@ degrees 7. training image +ead
a. 7. Figure 11 (ming). Recognition *it+ %arying illuminancefalseD a. test image lig+t mo%ed 7y 9 degrees 7. training image ( foof returned 7y t+e system +ead
Recognition !it" (ar-ing "ead &cale9 Eac" training image A!it" full "ead &cale "a& t!o corre&ponding te&t image&%one !it" a medium "ead &cale and t"e ot"er !it" a &mall one, a& &"o!n in Figure )' Ht"er condition&, &uc" a& lig"ting and "ead tilt, remain t"e &ame a& in t"e training image' ."e e6periment re&ult& are &"o!n in .a#le 3' )a7le !> Recognition *it+ %arying +ead scale
Num#er of te&t image& Num#er of truepo&iti(e identification& Num#er of fal&epo&iti(e identification& Num#er of fal&enegati(e identification&
I G )G 0
Figure 1$. )raining image and test images *it+ %arying +ead scale. a. training image 7. test image 1> medium +ead scale c. test image $> small +ead scale
Figure& )3 and )I illu&trate a truepo&iti(e e6ample and a fal&epo&iti(e one re&pecti(el-'
a.
7. Figure 1! (stan). Recognition *it+ %arying +ead scalesuccessD a. test image 1 medium scale 7. test image $ small scale c. training image full scale
a. 7. Figure 1- ( pascal . Recognition *it+ %arying +ead scalefalseD a. test image medium scale 7. training image (robert full scale
E6periment re&ult &ummar-9 From t"e e6periment& performed, a fairl- good recognition rate A%&'() i& o#tained !it" (ar-ing illuminance, an accepta#le rate A%%'() !it" different "ead tilt&, and a poor one A&'() !it" (ar-ing "ead &cale' ?o!e(er, it i& a long !a- to go #efore !e can confidentldra! a conclu&ion on t"e roug"ne&&8&en&iti(it- of t"e eigenface& recognition approac" to t"o&e condition&' Large ca&e &tud- need& carr-ing out in t"e &en&e t"at9 A) a large training &et i& re7uired, !"ic" con&i&t& of a large group of people, eac" "a(ing &e(eral face image& in t"e data#a&e: A numerou& te&t& are nece&&ar-, !it" face image& of people !"o are or aren=t in t"e data#a&e: A3 "o! doe& t"e &-&tem perform under com#ination& of condition c"ange&, e'g' &imultaneou& c"ange& in "ead tilt and illuminance, etc' ."re&"olding i&&ue i& not addre&&ed in 1)2' et, it doe& affect t"e performance of t"e algorit"m' Larger t"re&"old (alue lead& to lo!er fal&enegati(e rate, #ut "ig"er fal&e po&iti(e rate: and (ice (er&a' ;n ot"er !ord&, a good c"oice of t"re&"old (alue could !ell #alance fal&enegati(e and fal&epo&iti(e rate&, t"u& ma6imie good recognition rate'
2. Conclusion n eigenface&#a&ed face recognition approac" !a& implemented in MatLa#' ."i& met"od repre&ent& a face #- projecting original image& onto a lo!dimen&ional linear &u#&pace%‘face &pace=, defined #- eigenface&' ne! face i& compared to /no!n face cla&&e& #- computing t"e di&tance #et!een t"eir projection& onto face &pace' ."i& approac" !a& te&ted on a num#er of face image& do!nloaded from 1))2' Fairl- good recognition re&ult& !ere o#tained'
Hne of t"e major ad(antage& of eigenface& recognition approac" i& t"e ea&e of implementation' Fut"ermore, no /no!ledge of geometr- or &pecific feature of t"e face i& re7uired: and onl- a &mall amount of !or/ i& needed regarding preproce&&ing for ant-pe of face image&' ?o!e(er, a fe! limitation& are demon&trated a& !ell' Fir&t, t"e algorit"m i& &en&iti(e to "ead &cale' econd, it i& applica#le onl- to front (ie!&' ."ird, a& i& addre&&ed in 1)2 and man- ot"er face recognition related literature&, it demon&trate& good performance onlunder controlled #ac/ground, and ma- fail in natural &cene&' .o impro(e t"e performance of t"e eigenface recognition approac", a couple of t"ing& can #e done' A) .o reduce t"e fal&epo&iti(e rate, !e can ma/e t"e &-&tem return a num#er of candidate& from t"e e6i&ting face cla&&e& in&tead of a &ingle face cla&&' nd t"e remaining !or/ i& left to "uman' A Regarding t"e pattern (ector repre&enting a face cla&&, !e can ma/e eac" face cla&& con&i&t of &e(eral pattern (ector&, eac" con&tructed from a face image of t"e &ame indi(idual under a certain condition, rat"er t"an ta/ing t"e a(erage of t"e&e (ector& to repre&ent t"e face cla&&'.
2I. References )' 4Eigenface& for recognition, M' .ur/ and ' Pentland, *ournal of Cogniti$e Neuroscience+ $ol,+ No,%+ %..% ' 4Face recognition u&ing eigenface&, M' .ur/ and ' Pentland, /roc, 0111 Conf, on Computer 2ision and /attern 3ecognition, page& 5+5*), )**) 3' 4Face recognition for &mart en(ironment&, ' Pentland and .' C"oud"ur-, Computer+ 2ol, 0ss,(+ 4eb, (555 I' 4Face recognition9 Feature& (er&u& template&, R' Brunelli and .' Poggio, 0111 Trans, /attern Anal6sis and Machine 0ntelligence, %7"%5#8 %5)(-%57(+ %.. 5' 4?uman and mac"ine recognition of face&9 &ur(e-, R' C"ellappa, C' L' >il&on, and ' iro"e-, /roc, of 0111 , $olume 9+ pages &57-&)5+ %..7 ' 4Eigenface& (&' fi&"erface&9 Recognition u&ing cla&& &pecific linear projection, P' N' Bel"umeur, U' P' ?e&pan"a, and D' U' Vriegman, 0111 Trans, /attern Anal6sis and Machine 0ntelligence+ %."&%%-&(5+ %..& G' 4;llumination cone& for recognition under (aria#le lig"ting9 Face&, ' ' @eorg"iade&, D' U' Vriegman, and P' N' Bel"umeur, /roc, 0111 Conf, on Computer 2ision and /attern 3ecognition+ pages 7(-7.+ %..9 +' 4?uman face &egmentation and identification, ' ' iro"e-, Technical 3eport CA3-T3-:.7+ Center for Automation 3esearch+ ;ni$ersit6 of Mar6land+ College /ark+ MD+ %.. *' 4utomatic recognition and anal-&i& of "uman face& and facial e6pre&&ion&9 &ur(e-, ' amal and P' ' ;-engar, /attern 3ecognition+ (7"%#8 :7-&&+ %..( )0' 4Lo! dimen&ional procedure for t"e c"aracteriation of "uman face&, iro(ic", L' and Vir#-, M, *ournal of the
)ask Performed
Literature earc" Programming Po!erPoint lide& Report H(erall
Min Luo 50 +0 G0 +0 G0
u!apat Panitc"o# 50 0 30 0 30
