Machine Learning - Coursera About this course: Machine learning is the science of getting computers to act without being explicitly programmed. In the past decade, machine learning has given us self-driving cars, practical speech recognition, effective web search, and a vastly improved understanding of the human genome. Machine learning is so pervasive today that you probably use it dozens of times a day without knowing it. Many Man y researchers also think it is the best way to make progress towards human-level AI. In this class, you will learn about the most effective machine learning techniues, and gain practice implementing them and getting them to work for yourself. More importantly, importantly, you!ll learn about not only the theoretical underpinnings of learning, but also gain the practical know-how needed to uickly and powerfully apply these techniues to new problems. "inally, you!ll learn about some of #ilicon $all $alley!s ey!s best practices in innovation as it pertains to machine learning and AI. AI. %his course provides a broad introduction to machine learning, datamining, and statistical pattern recognition. %opics %opics include& 'i( #upervised learning 'parametric)non-parametric algorithms, support vector machines, kernels, neural networks(. 'ii( *nsupervised learning 'clustering, dimensionality reduction, recommender systems, deep learning(. 'iii( +est practices in machine learning 'bias)variance theory innovation process in machine learning and AI(. %he course will also draw from numerous case studies and applications, so that you!ll also learn how to a pply learning algorithms to building smart robots 'perception, control(, text understanding 'web search, anti-spam(, computer vision, medical informatics, audio, database mining, and other areas.
Created by: #tanford *niversity #yllabus WEEK 1 Introduction elcome e lcome to Machine earning/ In this module, we introduce the core idea of teaching a computer to learn concepts using data0without being explicitly programmed. %he 1ourse iki is under construction. 2lease visit the resources tab for the most co mplete and up-... 3 videos, 45 readings Graded: Introduction Linear Regression with One ariab!e inear regression predicts a real-valued output based on an input value. e discuss the application of linear regression to housing price prediction, present the notion of a cost function, and introduce the gradient descent method for learning. 6 videos, 7 readings ariable Graded: inear 8egression with 9ne $ariable Linear A!gebra Re"iew %his optional module provides a refresher on linear a lgebra concepts. +asic understanding of linear algebra is necessary for the rest of the course, especially as we begin to cover models with multiple variables. : videos, 4 reading, 4 reading WEEK # Linear Regression with Mu!ti$!e ariab!es
hat if your input has more than one o ne value; In this module, we show how linear regression can be extended to accommodate multiple input features. e e also discuss best practices for implementing linear regression. 7 videos, 4: readings Graded: inear 8egression with Multiple $ariables $ariables Octa"e%Mat!ab &u &utoria! toria! %his course includes programming assignments designed to help you understand how to implement the learning algorithms in practice. %o complete the programming assignments, you will need to use 9ctave or MA%A+. %his %his module introduces 9ctave)Matlab and shows yo... : videos, 4 reading Graded: 9ctave)Matlab %utorial WEEK ' Logistic Regression ogistic regression is a method for classifying data into discrete outcomes. "or exa mple, we might use logistic regression to classify an email as spam or not spam. In this module, we introduce the notion of classification, the cost function for logistic regr... 6 videos, 7 readings Graded: ogistic 8egression Regu!ari(ation Machine learning models need to generalize well to new examples that the model has not seen in practice. In this module, we introduce regularization, which helps prevent models from overfitting the training data. 3 videos, < readings Graded: 8egularization WEEK ) *eura! *etwor+s: Re$resentation =eural networks is a model inspired by how the brain works. It is widely widely used today in many applications& when your phone interprets and understand your voice commands, it is likely that a neural network is helping to understand your speech when you cash a ch... 6 videos, : readings Graded: =eural =etworks& 8epresentation WEEK , *eura! *etwor+s: Learning In this module, we introduce the backpropagation back propagation algorithm that is used to help learn parameters for a neural network. At the end of this module, you will be implementing your own neural network for digit recognition. 7 videos, 7 readings Graded: =eural =etworks& earning WEEK Ad"ice .or A$$!ying Machine Learning Applying machine learning in practice is not always straightforward. In this module, we share best practices for applying machine learning in practice, and discuss the best ways to evaluate performance of the learned models. 6 videos, > readings Graded: Advice for Applying Machine earning Machine Learning /yste0 esign
%o optimize a machine learning algorithm, you?ll need to first understand where the biggest improvements can be made. In this module, we discuss how to understand the performance of a machine learning system with multiple parts, and also how to deal with skewe... < videos, 4 reading Graded: Machine earning #ystem @esign WEEK 2 /u$$ort ector Machines #upport vector machines, or #$Ms, is a machine learning algorithm for classification. e introduce the idea and intuitions behind #$Ms and discuss how to use it in practice. : videos, 4 reading Graded: #upport $ector Machines WEEK 3 4nsu$er"ised Learning e use unsupervised learning to build models that help us understand our data better. e discuss the k-Means algorithm for clustering that enable us to learn groupings of unlabeled data points. < videos, 4 reading Graded: *nsupervised earning i0ensiona!ity Reduction In this module, we introduce 2rincipal 1omponents Analysis, and show how it can be used for data compression to speed up learning algorithms as well as for visualizations of complex datasets. 6 videos, 4 reading Graded: 2rincipal 1omponent Analysis WEEK 5 Ano0a!y etection iven a large number of data points, we may sometimes want to figure out which ones vary significantly from the average. "or example, in manufacturing, we may want to detect defects or anomalies. e show how a dataset can be modeled using a aussian distributi... 7 videos, 4 reading Graded: Anomaly @etection Reco00ender /yste0s hen you buy a product online, most websites automatically recommend other products that you may like. 8ecommender systems look at patterns of activities between different users and different products to produce these recommendations. In this module, we introd... : videos, 4 reading Graded: 8ecommender #ystems WEEK 16 Large /ca!e Machine Learning Machine learning works best when there is an abundance of data to leverage for training. In this module, we discuss how to apply the machine learning algorithms with large datasets. : videos, 4 reading Graded: arge #cale Machine earning WEEK 11 A$$!ication E7a0$!e: 8hoto OCR
Identifying and recognizing obBects, words, and digits in an image is a challenging task. e discuss how a pipeline can be built to tackle this problem and how to analyze and improve the performance of such a system. < videos, 4 reading Graded: Application& 2hoto 918
Intro to Machine Learning94dacity Lesson 1 Welcome to Machine Learning •
Learn what Machine Learning is and meet Sebastian Thrun!
•
Find out where Machine Learning is applied in Technology and Science.
Lesson 2 Naive Bayes •
Use Naie ayes with sci"it learn in python.
•
Splitting data between training sets and testing sets with sci"it learn.
•
#alculate the posterior probability and the prior probability o$ simple distributions.
Lesson % Support Vector Machines •
Learn the simple intuition behind Support &ector Machines.
•
'mplement an S&M classi(er in S)Learn*sci"it+learn.
•
'denti$y how to choose the right "ernel $or your S&M and learn about ,F and Linear )ernels.
Lesson Decision Trees •
•
#ode your own decision tree in python. Learn the $ormulas $or entropy and in$ormation gain and how to calculate them.
•
'mplement a mini proect where you identi$y the authors in a body o$ emails using a decision tree in /ython.
Lesson 0 Choose your own Algorithm •
ecide how to pic" the right Machine Learning lgorithm among )+Means3 daboost3 and ecision Trees.
Lesson 4 Datasets and uestions •
•
pply your Machine Learning "nowledge by loo"ing $or patterns in the 5nron 5mail ataset. 6ou7ll be inestigating one o$ the biggest $rauds in merican history!
Lesson 8 !egressions •
•
•
Understand how continuous superised learning is di9erent $rom discrete learning. #ode a Linear ,egression in /ython with sci"it+learn. Understand di9erent error metrics such as SS53 and , S:uared in the conte;t o$ Linear ,egressions.
Lesson < "utliers •
•
•
,emoe outliers to improe the :uality o$ your linear regression predictions. pply your learning in a mini proect where you remoe the residuals on a real dataset and reimplement your regressor. pply your same understanding o$ outliers and residuals on the 5nron 5mail #orpus.
Lesson = Clustering •
•
'denti$y the di9erence between Unsuperised Learning and Superised Learning. 'mplement )+Means in /ython and Sci"it Learn to (nd the center o$ clusters.
•
pply your "nowledge on the 5nron Finance ata to (nd clusters in a real dataset.
Lesson 1> #eature Scaling •
•
Understand how to preprocess data with $eature scaling to improe your algorithms. Use a min m; scaler in s"learn.
'ntroduction to Machine Learning ? Face etection in /ython@UdemyA 8euirements •
+asic python
@escription %his course is about the fundamental concepts of machine learning, focusing on neural networks, #$M and decision trees. %hese topics are getting very hot nowadays because these learning algorithms can be used in several fields from software engineering to investment banking. earning algorithms can recognize patterns which can help detect cancer for example or we may construct algorithms that can have a very good guess about stock prices movement in the market. In each section we will talk about the theoretical background for all of these algorithms then we are going to implement these problems together. %he first chapter is about regression& very easy yet ver y powerful and widely used machine learning techniue. e will talk about =aive +ayes classification and tree based algorithms such as decision trees and random forests. %hese are more sophisticated algorithms, sometimes works, sometimes not. %he last chapters will be about #$M and =eu ral =etworks& the most important approaches in machine learning. ho is the target audience; %his course is meant for newbies who are not familiar with machine learning or students looking for a uick refresher 1urriculum for %his 1ourse 4. Introduction a. Introduction b. Introduction to machine learning
>. 8egression a. inear regression introduction b. inear regression example c. ogistic regression introduction d. 1ross validation e. ogistic regression example I - sigmoid function f. ogistic regression example II g. ogistic regression example III - credit scoring C. D-=earest =eighbor 1lassifier a. D-nearest neighbor introduction b. D-nearest neighbor introduction - normalize data c. D-nearest neighbor example I d. D-nearest neighbor example II 3. =aive +ayes 1lassifier a. =aive +ayes introduction b. =aive +ayes example I c. =aive +ayes example II - text clustering <. #upport $ector Machine '#$M( a. #upport vector machine introduction b. #upport vector machine example I c. #upport vector machine example II - character recognition :. %ree +ased Algorithms a. @ecision trees introduction b. @ecision trees example I c. @ecision trees example II - iris data d. 2runing and bagging e. 8andom forests introduction f. +oosting g. 8andom forests example I h. 8andom forests example II - enhance decision trees 6. 1lustering a. 2rincipal component analysis introduction b. 2rincipal component analysis example c. D-means clustering introduction d. D-means clustering example e. @+#1A= introduction f. Eierarchical clustering introduction g. Eierarchical clustering example 7. =eural =etworks
a. b. c. d. e. f. g. h. i. B.
=eural network introduction "eedfordward neural networks %raining a neural network Frror calculation radients calculation +ackpropagation Applications of neural networks @eep learning =eural network example I - G98 problem =eural network example II - face recognition
H. "ace @etection a. "ace detection introduction b. Installing 9pen1$ c. 1ascade1lassifier d. 1ascade1lassifier parameters e. %uning the parameters 45. 9utro a. "inal words 44. #ource 1ode @ata a. #ource code 1#$ files b. @ata c. #lides d. 1oupon codes - get any of my courses for a discounted price
#S ->%*820B Foundations o$ Machine Learning @''T+A
Course Description 1# 35C)6>< provides a broad introduction to machine learning and various fields of application. %he course is designed in a way to build up from root level. &o$ics inc!ude: •
Superised #lassi(cation @perceptron3 support ector machine3 loss $unctions3 "ernels3 neural networ"s and deep learningA
•
Superised ,egression @Least s:uare regression3 bayes linear regressionA
•
Unsuperised classi(cation @clustering3 e;pectation ma;imiCationA
•
'ntroduction to learning theory @bias*ariance tradeo9sA.
The course will discuss the application o$ machine learning in deanagari script recognition which is a deeloping (eld in the machine learning community. Identi.ication
1# 6><& "oundations of machine learning escri$tion
8emedial co-reuisite& Mathematical foundations '#eparately proposed by 2rof. #aketh =ath( 8ecommended parallel courses& 1#65H '1onvex optimization( 1ourse 1ontent & #upervised learning& decision trees, nearest neighbor classifiers, generative classifiers like naive +ayes, linear discriminate analysis, loss regularization framework for classification, #upport vector Machines 8egression methods& least-suare regression, kernel regression, regression trees *nsupervised learning& k-means, hierarchical, FM, non-negative matrix factorization, rate distortion theory. Re.erences
4. Eastie, %ibshirani, "riedman %he elements of #tatistical earning #pringer $erlag. >. 2attern recognition and machine learning by 1hristopher +ishop. C. #elected papers. ;o0e 8age
http&))www.cse.iitb.ac.in)Jsunita)cs6>< 8rere
=)A
'ntroduction to Machine Learning
bout The #ourse %his course provides a concise introduction to the fundamental concepts in machine learning and popular machine learning algorithms. e will cover the standard and most popular supervised learning algorithms including linear regression, logistic regression, decision trees, k-nearest neighbour, an introduction to +ayesian learning and the na Kve +ayes algorithm, support vector machines and kernels and neural networks with an introduction to @eep earning. e will also cover the basic clustering algorithms. "eature reduction methods will also be discussed. e will introduce the basics of computational learning theory. In the course we will discuss various issues related to the application of machine learning algorithms. e will discuss hypothesis space, overfitting, bias and variance, tradeoffs between representational power and learnability, evaluation strategies and cross-validation. %he course will be accompanied b y hands-on problem solving with programming in 2ython and some tutorial sessions.
'ntended udience Flective course * or 2 +F)MF)M#)M#c)2h@
/re+re:uisites +asic programming skills 'in 2ython(, algorithm design, basics of probability statistics
'ndustry Support + List o$ #ompanies*'ndustry that will ,ecogniCe*alue this online course @ata science companies and many other industries value machine learning skills.
#ourse 'nstructor
/udeshna /ar+ar is a 2rofessor and currently the Eead in the @epartment of 1omputer #cience and Fngineering at II% Dharagpur. #he completed her +.%ech. in 4H7H from II% Dharagpur, M# from *niversity of 1alifornia, +erkeley, and 2h@ from II% Dharagpur in 4HH<. #he served briefly in the faculty of II% uwahati and at II% Danpur before Boining II% Dharagpur in 4HH7. Eer research interests are in Machine earning, =atural anguage 2rocessing, @ata and %ext Mining.
%he %eaching Assistants of this course are Anirban #antara and Ayan @as, both of whom are 2h@ students in 1omputer #cience Fngineering @epartment, II% Dharagpur. %hey will take active part in the course especially in running demonstration and programming classes as well as tutorial classes.
#ourse layout
Week 1:
'ntroductionB asic de(nitions3 types o$ learning3 hypothesis space and inductie bias3 ealuation3 cross+alidation Week 2:
Linear regression3 ecision trees3 oer(tting Week 3:
'nstance based learning3 Feature reduction3 #ollaboratie (ltering based recommendation Week 4:
/robability and ayes learning Week 5:
Logistic ,egression3 Support &ector Machine3 )ernel $unction and )ernel S&M Week 6:
Neural networ"B /erceptron3 multilayer networ"3 bac"propagation3 introduction to deep neural networ" Week 7:
#omputational learning theory3 /# learning model3 Sample comple;ity3 & # imension3 5nsemble learning Week 8:
#lusteringB "+means3 adaptie hierarchical clustering3 Daussian mi;ture model suggested reading 4. Machine earning. %om Mitchell. "irst Fdition, Mcraw- Eill, 4HH6. >. Introduction to Machine earning Fdition >, by Fthem Alpaydin
More details about the course Course url: httpsB**onlinecourses.nptel.ac.in*noc14Ecs1< Course duration : >< wee"s Start date and end date of course: 1< uly 2>14 + >= September 2>14 Dates of exams : 1< September 2>14 ? 20 September 2>14 Time of exam : 2pm + 0pm
Final List o$ e;am cities will be aailable in e;am registration $orm. 5;am registration url + Gill be announced shortly xam !ee: The online registration $orm has to be (lled and the certi(cation e;am $ee o$ appro;imately "s 1###$non%&ro'rammin'()125#$&ro'rammin'(needs to be paid. certi(cate F-1ertificate will be given to those who register and write the exam. 1ertificate will have your name, photograph and the score in the final exam. It will have the logos of =2%F and II%
Dharagpur. It will be e-verifiable at nptel.ac.in)noc.
'ntroduction to Machine Learning A=O4& &;E CO4R/E With the increased availability of data from varied sources there has been increasing attention paid to the various data driven disciplines such as analytics and machine learning. In this course we intend to introduce some of the basic concepts of machine learning from a mathematically well motivated perspective. We will cover the different learning paradigms and some of the more popular algorithms and architectures used in each of these paradigms. INTENDED AUDIENCE
This is an elective course. Intended for senior UG/PG students. B/!/!"/Ph# PRE-REQUISITES
We will assume that the students $now programming for some of the assignments.If the students have done introdcutory courses on probability theory and linear algebra it would be helpful. We will review some of the basic topics in the first two wee$s as well. INDUSTRIES THAT WILL VALUE THIS
%ny company in the data analytics/data science/big data domain would value this course. C*+"S ,-ST"+CT*"
Prof. Ravindran is
currently an associate pro$essor in #omputer Science at ''T Madras. He has nearly two decades o$ research e;perience in machine learning and speci(cally rein$orcement learning. #urrently his research interests are centered on learning $rom and through interactions and span the areas o$ data mining3 social networ" analysis3 and rein$orcement learning.
C*+"S ./0*+T Wee$ &' Introductory Topics Wee$ (' )inear *egression and +eature "election Wee$ ,' )inear -lassification Wee$ ' "upport ector !achines and %rtificial 0eural 0etwor$s Wee$ 1' Bayesian )earning and #ecision Trees Wee$ 2' valuation !easures Wee$ 3' 4ypothesis Testing Wee$ 5' nsemble !ethods Wee$ 6' -lustering Wee$ &7' Graphical !odels Wee$ &&' )earning Theory and 8pectation !a8imi9ation Wee$ &(' Introduction to *einforcement )earning
#erti(cation 5;am %he exam is optional. Fxams will be on >3 April >54: and C5 April >54:. %ime& >pm-
51ity-list.pdf 8egistration url& Announcements will be made when the registration form is open for registrations. %he online registration form has to be filled and the certification exam fee of approximately 8s 4555 needs to be paid.
#erti(cate 1ertificate will be given to those who register and write the exam. 1ertificate will have your name, photograph and the score in the final exam. It will have the logos of =2%F and II% Madras. It will be e-verifiable at nptel.ac.in)noc.
S+STD "/D,- &. T. 4astie: *. Tibshirani: ;. +riedman. The lements of "tatistical )earning: (e: (775. (. -hristopher Bishop. Pattern *ecognition and !achine )earning. (e.
''T Madras #S0>11B 'ntroduction to Machine Learning
Home I ,esearch ? /ublications I Teaching I Students I #& I #ontact Sr Date -o
.ecture Contents
"eference
1
ug 13 2>11
'ntroduction to machine learning
#hapter 1 $rom Machine Learning by Tom Mitchell
2
ug 23 2>11
'ntroduction to machine learning
#hapter 1 $rom Machine Learning by Tom Mitchell
%
ug -3 2>11
Jeriew o$ target $unction representations
#hapter 1 $rom Machine Learning by Tom Mitchell
-
ug 03 2>11
Hypothesis class3 ersion space
#hapter 1 and 2 $rom Machine Learning by Tom Mitchell
0
ug <3 Types o$ ML techni:ues3 hypothesis 2>11 selection through cross alidation
#hapter 2 $rom 'ntroduction to Machine Learning by 5them lppaydin
4
ug =3 2>11
#hapter 2 $rom 'ntroduction to Machine Learning by 5them lppaydin
8
ug 113 K? on oer and under+(tting3 bias+ 2>11 ariance3 ataB types o$ $eatures3 data normaliCation
<
ug 123 ias ariance trade+o9 using regression 2>11 e;ample
=
ug 143 #orrelation3 coariance3 Mahalanobis 2>11 distance
Noise3 bias+ariance trade+o93 under+ (tting and oer+(tting concepts
#hapter 2 $rom /rinciples o$ ata Mining by aid Hand et al.
#hapter 2 $rom /rinciples o$ ata Mining by aid Hand et al.
1> ug 1<3 Mahalanobis distance3 Min"ows"i 2>11 distance3 distance metric3 accard coecient3 missing alues3 $eature trans$ormations
#hapter 23 % $rom /rinciples o$ ata Mining by aid Hand et al.
11 ug 1=3 Deometrical interpretation o$ 5uclidean3 2>11 Mahalanobis distance3 dealing with uncertainty
#hapter - $rom /rinciples o$ ata Mining by aid Hand et al.
12 ug 223 Ma;imum li"eliHood estimation @ML5A 2>11 theory and e;ample using binomial distribution
#hapter - $rom /rinciples o$ ata Mining by aid Hand et al.
1% ug 2%3 Ma;imum li"eliHood estimation @ML5A o$ #hapter - $rom /rinciples o$
2>11
uniariate Daussian3 generatie s discriminatie models
ata Mining by aid Hand et al.
1- ug 203 Ma;imum li"elihood estimation o$ #hapter - $rom /rinciples o$ 2>11 biariate Daussian distribution3 sucient ata Mining by aid Hand statistics et al. 10 ug 243 ayesian Learning 2>11
#hapter 2 $rom /attern ,ecognition and Machine Learning by #hristopher M. ishop
CS5#11 % acine .earnin' Course Data $ Sylla%us$ asic ats B /robability3 Linear lgebra3 #one; JptimiCation ack'round B Statistical ecision Theory3 ayesian Learning @ML3 M/3 ayes
estimates3 #onugate priorsA "e'ression B Linear ,egression3 ,idge ,egression3 Lasso Dimensionalit "eduction B /rincipal #omponent nalysis3 /artial Least S:uares Classication B Linear #lassi(cation3 Logistic ,egression3 Linear iscriminant nalysis3 Kuadratic iscriminant nalysis3 /erceptron3 Support &ector Machines )ernels3 rti(cial Neural Networ"s ac"/ropagation3 ecision Trees3 ayes Jptimal #lassi(er3 Naie ayes. aluation measures B Hypothesis testing3 5nsemble Methods3 agging daboost Dradient oosting3 #lustering3 )+means3 )+medoids3 ensity+based Hierarchical3 Spectral iscellaneous to9ics B 5;pectation Ma;imiCation3 DMMs3 Learning theory 'ntro to ,ein$orcement Learning ra9ical odelsB ayesian Networ"s.
Machine Learning #S881 utumn 2>14
'nstructorB /iyush ,aiB @oceB )+%1=3 emailB piyush T cse JT iit" JT ac JT inA Jce HoursB Tuesday 12+1pm @or by appointmentA )/ !orum: /iaCCa @please registerA #lass LocationB L+14 @lecture hall comple;A TimingsB GF 4B>>+8B%>pm
ac"ground and #ourse escription Machine Learning is the discipline o$ designing algorithms that allow machines @e.g.3 a computerA to learn patterns and concepts $rom data without being e;plicitly programmed. This course will be an introduction to the design @and some analysisA o$ Machine Learning algorithms3 with a modern outloo"3 $ocusing on the recent adances3 and e;amples o$ real+world applications o$ Machine Learning algorithms. This is supposed to be the (rst @introA course in Machine Learning. No prior e;posure to Machine Learning will be assumed. t the same time3 please be aware that this is NJT a course about tool"its*so$tware*/'s used in applications o$ Machine Learning3 but rather on the principles and $oundations o$ Machine Learning algorithms3 deling deeper to understand what goes on under the hood3 and how Machine Learning problems are $ormulated and soled.
/re+re:uisites MSJ2>1*e:uialent3 #S21>*5SJ211*5SJ2>8O bility to program in MTL*Jctae. 'n some cases3 pre+re:uisites may be waied @will need instructor7s consentA.
Drading There will be - homewor" assignments @total ->PA which may include a programming component3 a mid+term @2>PA3 a (nal+e;am @2>PA3 and a course proect @2>PA
,e$erence materials There will not be any dedicated te;tboo" $or this course. 'n lieu o$ that3 we will hae lecture slides*notes3 monographs3 tutorials3 and papers $or the topics that will be coered in this course. Some recommended3 although not re:uired3 re$erence boo"s are listed below @in no particular orderAB •
•
•
•
•
Treor Hastie3 ,obert Tibshirani3 erome Friedman3 The 5lements o$ Statistical Learning3 Springer3 2>>= @$reely aailable onlineA Hal aumQ '''3 #ourse in Machine Learning3 2>10 @in preparationO most chapters $reely aailable onlineA )ein Murphy3 Machine LearningB /robabilistic /erspectie3 M'T /ress3 2>12 #hristopher ishop3 /attern ,ecognition and Machine Learning3 Springer3 2>>8. Shai Shale+ShwartC and Shai en+aid. Understanding Machine LearningB From Theory to lgorithms3 #ambridge Uniersity /ress3 2>1-
Schedule @TentatieA Dat To9ics e
"eadin's)"eferences
Deadli Slides)-o nes tes
#ourse Logistics and uly 'ntroduction to Machine 2< Learning
Linear lgebra reiew3 /robability reiew3 Matri; #oo"boo"3 MTL reiew3 RM103 RLH10
slides
Su9erised .earnin' Learning by #omputing ug istancesB istance $rom istance $rom Means3 #'ML % Means and Nearest #hapter 2 Neighbors
slides
Learning by s"ing ug KuestionsB ecision Tree oo" #hapter3 'n$o Theory 0 based #lassi(cation and notes T + isual illustration ,egression
slides
JptionalB Some notes3 Some ug Learning as JptimiCation3 use$ul resources on 1> Linear ,egression optimiCation $or ML
slides
Learning ia /robabilistic ug Murphy @ML//AB #hapter 8 ModelingB /robabilistic 12 @sections 8.1+8.0A Linear ,egression
slides
Learning ia /robabilistic ug Murphy @ML//AB #hapter < ModelingB Logistic and 18 @sections <.1+<.%A So$tma; ,egression
slides
Jnline Learning ia ug Murphy @ML//AB #hapter < Stochastic JptimiCation3 1= @section <.0A /erceptron
slides
Learning Ma;imum+ 'ntro to S&M3 Gi"ipedia 'ntro to ug Margin HyperplanesB S&M3 JptionalB danced 'ntro 2Support &ector Machines to S&M3 S&M Solers
slides
ug Nonlinear Learning with 24 )ernels
#'ML #hapter = @section =.1 and =.-A3 Murphy @ML//AB #hapter 1- @up to section 1-.-.%A
slides
+nsu9erised .earnin' ishop @/,MLAB Section =.1. ug ata #lustering3 )+means Jptional readingB ata %1 and )ernel )+means clusteringB 0> years beyond "+ means
HG 1 ue
slides
Linear imensionality Sept ,eductionB /rincipal 2 #omponent nalysis
ishop @/,MLAB Section 12.1. Jptional readingB /# tutorial paper
/# @Grap+upA and Sept Nonlinear imensionality Jptional readingB )ernel /# 8 ,eduction ia )ernel /# Sept Matri; FactoriCation and 21 Matri; #ompletion
Jptional ,eadingB Matri; FactoriCation $or ,ecommender Systems3 Scalable MF
Sept 'ntroduction to 2% Deneratie Models
slides
slides
slides
slides
Deneratie Models $or Sept #lusteringB DMM and 24 'ntro to 5M
ishop @/,MLAB Section =.2 and =.% @up to =.%.2A
slides @notesA
5;pectation Sept Ma;imiCation and 2< Deneratie Models $or im. ,eduction
ishop @/,MLAB Section =.% @up to =.%.2A and =.-
slides
Deneratie Models $or Jct im. ,eductionB 0 /robabilistic /# and Factor nalysis
ishop @/,MLAB Section 12.2 @up HG 2 to 12.2.2A. Jptional readingB ue Mi;tures o$ //#
slides
/ssorted To9ics /ractical 'ssuesB Model*Feature Selection3 Jct Jn 5aluation and Model 5aluating and 1= Selection ebugging ML lgorithms
slides
Jct 'ntroduction to Learning 2- Theory
Jptional @but recommendedA Mitchell ML #hapter 8 @sections 8.1+8.%.13 section 8.- @up to 8.-.2AA
slides
Jct 5nsemble MethodsB 24 agging and oosting
#'ML #hapter 113 JptionalB rie$ 'ntro to oosting3 5;plaining daoost
slides
Jct Semi+superised 2< Learning
,eadingB rie$ SSL 'ntro3 JptionalB @somewhat old but
slides
recommendedA surey on SSL eep Learning @1AB Jptional ,eadingsB Feed$orward No HG % Feed$orward Neural Nets Neural Networ"s3 #onolutional 2 ue and #NN Neural Nets
slides
eep Learning @2AB No Models $or Se:uence ata @,NN and LSTMA and utoencoders
slides
Jptional ,eadingsB ,NN and LSTM3 Understanding LSTMs3 ,NN and LSTM ,eiew
No Learning $rom 0 'mbalanced ata Jnline Learning No @dersarial Model and = 5;pertsA
slides Jptional ,eadingB Foundations o$ ML @#hapter 8A
No Surey o$ Jther Topics 11 and #onclusions
slides
slides
Use$ul Lin"s + Machine Learning Summer Schools + Sci"it+LearnB Machine Learning in /ython + wesome Machine Learning @a comprehensie list o$ arious Machine Learning libraries and so$twaresA
,,Sc an'alore E6 #26 9':1
Machine Learning
shi"ani % Chiran>ib =hattacharyya % Indra>it =hattacharya
Introduction to machine learning. 1lassification& nearest neighbour, decision trees, perceptron, support vector machines, $1-dimension. 8egression& linear least suares regression, support vector regression. Additional learning problems& multiclass classification, ordinal regression, ranking. Fnsemble methods& boosting. 2robabilistic models& classification, regression, mixture models 'unconditional and conditional(, parameter estimation, FM algorithm. +eyond II@, directed graphical models& hidden Markov models, +ayesian networks. +eyond II@, undirected graphical models& Markov random fields, conditional random fields. earning and inference in +ayesian networks and M8"s& parameter estimation, exact inference 'variable elimination, belief propagation(, approximate inference 'loopy belief propagation, sampling(. Additional topics& semi-supervised learning, active learning, structured prediction. 8eferences&
•
+ishop. 1 M, 2attern 8ecognition and Machine earning. #pringer, >55:.
•
@uda, 8 9, Eart 2 F and #tork @ . 2attern 1lassification. iley-Interscience, >nd Fdition, >555.
•
Eastie %, %ibshirani 8 and "riedman , %he Flements of #tatistical earning& @ata Mining, Inference and 2rediction. #pringer, >nd Fdition, >55H.
•
Mitchell %, Machine earning. Mcraw Eill, 4HH6.
•
1urrent literature.
8rere
2robability and #tatistics 'or euivalent course elsewhere(. #ome background in linear algebra and optimization will be helpful.
•
,,T Deli #SL%-1B Fundamentals o$ Machine Learning
&eneral 'n(ormation ,nstructor: /arag Singla @emailB parags T cse.iitd.ac.inA
Class Timin's $Slot (:
Monday3 =B%>am + 1>B00am
•
•
Thursday3 =B%>am + 1>B00am
;enue:GS 1>1 @Gor"shop ,oom 1>1A harti 1>1 Teaching Assistants
-ame
mail
bhina )umar
cs0>=>2%1 T cse.iitd.ac.in
nu Dupta
agupta T cse.iitd.ac.in
rpit ain
cs0>=>2%4 T cse.iitd.ac.in
Happy Mittal
csC1%<2%% T cse.iitd.ac.in
Shubham Dupta
cs0>=>202 T cse.iitd.ac.in
Sudhanshu Se"har
cs0>=>200 T cse.iitd.ac.in
6amuna /rasad
yprasad T cse.iitd.ac.in
Announcements •
RThu Jct %1B ssignment 23 New ue ateB Monday No - @11B0> pmA.
•
RMon Sep %>B ssignment 2 is out! ue ateB Thursday Jct %1 @11B0> pmA.
•
•
•
RFri Sep 28B ssignment submission instructions hae been updated @See belowA. RGed Sep 20B ssignment 1 has been updated. New ue ateB Sunday Sep 2= @11B0> pmA. RGed Sep -B The enue $or the class on Thursday Sep 0 will be harti 1>1 @instead o$ GS 1>1A.
•
RSat ug 1>B ssignment 1 is out! ue ateB Sunday Sep 10 @11B0> pmA.
•
RGed ul %1B The course website is up3 (nally!
Course Content
Wee To9ic k
ook Ca9ters
1
'ntroduction
uda3 #hapter 1
23%
Linear and Logistic ,egression3 Daussian iscriminant nalysis
ishop3 #hapter %.13 -
lin+log+reg.pd$ 3 gda.pd$
-30
Support &ector Machines
ishop3 #hapter 8.1
sm.pd$
4
Neural Networ"s
Mitchell3 #hapter -
nnets.pd$ nnets+hw.pd$
8
ecision Trees
Mitchell3 #hapter %
dtrees.pd$
<3=
Naie ayes3 ayesian Statistics
Mitchell3
nb.pd$ 3 bayes.pd$
Su99lementar -otes
#hapter 4
#onugate /rior model.pd$
1>31 )+Means3 Daussian Mi;ture 1 Models3 5M
"means.pd$ gmm.pd$ em.pd$
12
/#
pca.pd$
1%
Learning Theory3 Model Selection
1-
pplication o$ ML to #rowdSourcing and NL/
Mitchell3 #hapter 8
theory.pd$ model.pd$ crowd+ml.pd$ nlp+ml.pd$
Additional !eading •
'nduction o$ ecision Tress )"riginal *aper on the 'D+ Algorithm %y !oss uinlan,
•
•
eyond 'ndependenceB #onditions $or the Jptimality o$ the Simple ayesian #lassi(er Uni(ed ias+&ariance ecomposition $or ero+Jne and S:uared Loss
!eview Material
Topic
-otes
/robability
prob.pd$
Linear lgebra
linalg.pd$
Daussian istribution
gaussians.pd$
#one; JptimiCation @1A
cone;+1.pd$
!e(erences
1. *attern !ecognition and Machine Learning- #hristopher ishop. First 5dition3 Springer3 2>>4. 2. *attern Classi.cation- ,ichard uda3 /eter Hart and aid Stoc". Second 5dition3 Giley+'nterscience3 2>>>. %. Machine Learning- Tom Mitchell. First 5dition3 McDraw+Hill3 1==8.
Assignment Su%mission 'nstructions
1. 6ou are $ree to discuss the problems with other students in the class. 6ou should include the names o$ the people you had a signi(cant discussion with in your submission. 2. ll your solutions should be produced independently without re$erring to any discussion notes or the code someone else would hae written. %. ll the programming should be done in MTL. 'nclude comments $or readability. -. #ode should be submitted using Sa"ai /age. 0. RUpdated Jctober %13 2>1%B #reate a separate directory $or each o$ the :uestions named by the :uestion number. For instance3 $or :uestion 13 all your submissions (les @code*graphs*write+upA should be put in the directory named K1 @and so on $or other :uestionsA. /ut all the Kuestion sub+ directories in a single top leel directory. This directory should be named as yourentrynumberE(rstnameElastname. For e;ample3 i$ your entry number is 2>>=anC80%0 and your name is Nilesh /atha"3 your submission directory should be named as 2>>=anC80%0EnileshEpatha". 6ou should Cip your directory and name the resulting (le as yourentrynumberE(rstnameElastname.Cip e.g. in the aboe e;ample it will be 2>>=anC80%0EnileshEpatha".Cip. This single Cip (le should be submitted online. 4. P $or each late day in submission. Ma;imum o$ 2 days late submissions are allowed. Assignments
1. ssignment 2 New ue ateB 11B0> pm3 Monday Noember -3 2>1%. atasetsB o
/roblem 1B :1Edata.Cip
o
/roblem 2B :2Edata.Cip
o
/roblem %B :%Edata.Cip
2. ssignment 1. New ue ateB Sunday September 2=3 2>1%. o
New Updated &ersion
o
Jriginal &ersion atasets
/roblem 1B :1;.dat :1y.dat
/roblem 2B :2;.dat :2y.dat
,,T =ara'9ur Machine Learning @#S4>>0>A Instructor& #ourangshu +hattacharya 1lass #chedule& F@'5H&C5-45&C5( , %E*8#'57&C5-5H&C5( , "8I'45&C5-44&C5( , "8I'44&C54>&C5( 1lassroom& 1#F-457 ebsite& http&))cse.iitkgp.ac.in)Jsourangshu)cs:55<5.html "irst Meeting& ednesday, >3th uly, at 5H&C5 am in 1#F-457. /y!!abus: =asic 8rinci$!es: Introduction, %he concept learning task. eneral-to-specific ordering of hypotheses. $ersion spaces. Inductive bias. Fxperimental Fvaluation& 9ver-fitting, 1ross$alidation. /u$er"ised Learning: @ecision %ree earning. Instance-+ased earning& k-=earest neighbor algorithm, #upport $ector Machines, Fnsemble learning& boosting, bagging. Artificial =eural =etworks& inear threshold units, 2erceptrons, Multilayer networks and back-propagation. 8robabi!istic Mode!s: Maximum ikelihood Fstimation, MA2, +ayes 1lassifiers =aive +ayes. +ayes optimal classifiers. Minimum description length principle. +ayesian =etworks, Inference in +ayesian =etworks, +ayes =et #tructure earning. 4nsu$er"ised Learning: D-means and Eierarchical 1lustering, aussian Mixture Models, FM algorithm, Eidden Markov Models. Co0$utationa! Learning &heory: probably approximately correct '2A1( learning. #ample complexity. 1omputational complexity of training. $apnik - 1hervonenkis dimension, 8einforcement earning. &e7tboo+s: •
•
Tom Mitchell. Machine Learning. McDraw Hill3 1==8. #hristopher M. ishop. /attern ,ecognition and Machine Learning. Springer 2>>4.
•
,ichard J. uda3 /eter 5. Hart3 aid D. Stor". /attern #lassi(cation. ohn Giley ? Sons3 2>>4.
