Basic Vision with LabVIEW Raul G. Longoria Updated Fall 2011
ME 144L – Prof. R.G. R.G. Longoria Longoria Dynamic Systems and Controls Laboratory
Department of Mechanical Engineering The University Univers ity of Texas Texas at Austin
Machine Vision Concepts* •
Mac Machine hine (or (or com compute puter) r) visi visio on has six six prin rincipa cipall are reas as 1. 2. 3. 4. 5. 6.
• •
Sensin sing – whic hich yi yields a vi visua sual im image Prep Pr epro roce cess ssin ing g – nois noisee redu reduct ctio ion, n, im image age enha enhanc ncem emen ent, t, etc etc.. Segm Segmen enta tati tion on – part partit itio ioni ning ng an imag imagee int into o objec objects ts of inte intere rest st Desc Descri ript ptio ion n – comp comput utat atio ion n of feat featur ures es for for dif diffe fere rent ntia iati ting ng amon among g type typess of objects Reco Recogn gnit itio ion n – iden identi tify fyin ing g objec objects ts (e.g (e.g., ., bolt bolt,, wrenc wrench, h, etc. etc.)) Inte Interpr rpret etat atio ion n – assi assign gnin ing g mean meanin ing g to to an an ens ensem embl blee of of rec recogn ogniz ized ed objects
Lev Levels els of proce rocess ssin ing g ar aree div divid ideed into into low low (1, (1, 2), 2), med mediu ium m (3,4,5), and high (6) In this this cour course se,, we we’ll ’ll pri prim mar aril ily y be be con conce cern rned ed with ith lo low-lev -level el vision, and will utilize some functions of medium-level vision.
ME 144L – Prof. R.G. R.G. Longoria Longoria Dynamic Systems and Controls Laboratory
Department of Mechanical Engineering The University Univers ity of Texas Texas at Austin
Machine Vision Concepts* •
LowLow-le leve vell vis visio ion n invo involv lves es proc proces esse sess con consi side dere red d ‘pri ‘prim mitiv itive’ e’ (or (or automatic) and requiring no ‘intelligence’ ‘intelligence’ (1,2). This could be thought of as analogous to how a human eye senses and adapts.
•
Med Medium ium-lev -level el visio ision n ext extra ract cts, s, char charac acte teri rizzes, and and lab labels els components in an image.
•
High High-l -lev evel el visi vision on re refe fers rs to proc proces esse sess that that atte attem mpt to emul emulat atee perception and cognition.
*From Fu, Gonzalez, and Lee, Robotics: Control, Sensing, Vision, and Intelligence, McGraw-Hill, New York, 1987.
ME 144L – Prof. R.G. R.G. Longoria Longoria Dynamic Systems and Controls Laboratory
Department of Mechanical Engineering The University Univers ity of Texas Texas at Austin
“To see is to think.” – Salvador Dali (and quoted by E. Dickmanns in his book Dynamic Vision for Perception and Control of Motion, Springer, 2007)
The Persistence of Memory, 1931 S. Dali
From Dickmanns (2007)
ME 144L – Prof. R.G. R.G. Longoria Longoria Dynamic Systems and Controls Laboratory
Department of Mechanical Engineering The University Univers ity of Texas Texas at Austin
LabVIEW-based Vision • LabVI abVIE EW Vis Visio ion n ena enabl bles es you you to to re read ad/c /cre reat atee image image files and provides means for managing those files analyz yzin ing g imag imagee • Ther Theree are are bui built lt-i -in n fun funct ctio ions ns (VIs (VIs)) for for anal files (select areas of interest, measure intensity, etc.) • It is neces necessar sary y to to also also have have LabV LabVIE IEW W IMAQ IMAQ softw softwar aree which enables you to acquire images from cameras. • In this this cour course se,, we we want want to demons demonstra trate te how how you you can can use these software tools to develop a simple visionbased measurement system, particularly for object motion. ME 144L – Prof. R.G. R.G. Longoria Longoria Dynamic Systems and Controls Laboratory
Department of Mechanical Engineering The University Univers ity of Texas Texas at Austin
Overview of LV-Based Vision Tools • Image data type • Analyzing images
Vision Utilities Image processing Machine vision
• Capturing images
ME 144L – Prof. R.G. R.G. Longoria Longoria Dynamic Systems and Controls Laboratory
Department of Mechanical Engineering The University Univers ity of Texas Texas at Austin
Analyzing Images • Vision Utilities – VIs VIs for for cr crea eati ting ng and and manipulating images, etc. • Image Processing – provi provides des ‘low ‘low lev level el’’ VIs VIs for analyzing images. • Machine Vision – group groupss many many prac practi tical cal VIs VIs for performing image analysis. For example, the “Count and Measure Objects” VI is found under this group. ME 144L – Prof. R.G. R.G. Longoria Longoria Dynamic Systems and Controls Laboratory
Department of Mechanical Engineering The University Univers ity of Texas Texas at Austin
Vision Utilities To create and manipulate images
• Imag Imagee mana managem gemen entt (c (cre reat ate, e, dispo dispose se,, etc etc.) .) • File handling • Image ma manipulation • Pixel ed editing • etc. • Best Best to lear learn n use use thro throug ugh h exa examp mple les. s. ME 144L – Prof. R.G. R.G. Longoria Longoria Dynamic Systems and Controls Laboratory
Department of Mechanical Engineering The University Univers ity of Texas Texas at Austin
Reading an Image File This VI block diagram opens an existing image file (e.g., a bitmap), reads the file, and then displays it.
On the Front Panel, place an ‘Image Display’ to get this terminal; then wire image data.
ME 144L – Prof. R.G. R.G. Longoria Longoria Dynamic Systems and Controls Laboratory
Department of Mechanical Engineering The University Univers ity of Texas Texas at Austin
Exam Exampl plee – look lookin ing g at an an ima image ge The simple VI in the previous slide can be used to open an image file.
This read-out indicates the size of the image (pixels). When moving the cursor around the image, the readout shows cursor (x,y) coordinates and the ‘intensity’ value at that location.
This is the ‘clamp’ example file provided in LabVIEW
ME 144L – Prof. R.G. R.G. Longoria Longoria Dynamic Systems and Controls Laboratory
Department of Mechanical Engineering The University Univers ity of Texas Texas at Austin
Some LV Machine Vision VIs • Sel Select ect Regi Region on of Int Inter eres estt • Light Meter • Coun Countt and and Meas Measur uree Obj Objects ects • You’ You’ll ll lear learn n how how to to use use some some of these these in the the lab. There are many others you can skim through to get an idea of what is available. ME 144L – Prof. R.G. R.G. Longoria Longoria Dynamic Systems and Controls Laboratory
Department of Mechanical Engineering The University Univers ity of Texas Texas at Austin
IMAQ Select Region of Interest You can focus on regions based on: point, line, rectangle, annulus
The output from this VI can be sent to other VIs that require that bounding information. ME 144L – Prof. R.G. R.G. Longoria Longoria Dynamic Systems and Controls Laboratory
Department of Mechanical Engineering The University Univers ity of Texas Texas at Austin
IMAQ Light Meter If you want to examine examine pixel intensity in a certain region, you need rectangle information.
IMAQ Select Rectangle Use to specify a rectangular area in the image. Rectangle coordinates are output and can be sent to next function.
Rectangle
Image ME 144L – Prof. R.G. R.G. Longoria Longoria Dynamic Systems and Controls Laboratory
Histogram data (send directly to a waveform graph) Department of Mechanical Engineering The University Univers ity of Texas Texas at Austin
Count and Measure Objects Another VI that needs rectangle information, and which is very useful for basic segmentation is the ‘Count and Measure Objects’ VI. This VI needs several inputs, as shown below.
NOTE: This VI requires that you convert conv ert the image to grayscale. ME 144L – Prof. R.G. R.G. Longoria Longoria Dynamic Systems and Controls Laboratory
Department of Mechanical Engineering The University Univers ity of Texas Texas at Austin
IMAQ Cast Image
Example usage shown below:
ME 144L – Prof. R.G. R.G. Longoria Longoria Dynamic Systems and Controls Laboratory
Department of Mechanical Engineering The University Univers ity of Texas Texas at Austin
Exampl Examplee – Findin Finding g Object Objectss and Intens Intensiti ities es
Image Rectangle Error
3 objects detected The limit on object size prevented the 3 larger objects in the ROI from being identified ME 144L – Prof. R.G. R.G. Longoria Longoria Dynamic Systems and Controls Laboratory
Department of Mechanical Engineering The University Univers ity of Texas Texas at Austin
Define ‘Threshold’
Bright objects have ‘high’ ‘h igh’ intensity values (e.g., 255 for 8-bit) Dark objects have ‘low’ intensity values (e.g., 0 for fo r 8-bit) These objects have an intensity of close to zero.
These objects have an intensity of close to 255.
For 8-bit image The ‘Threshold’ must often be specified as an input to some machine vision VIs. ME 144L – Prof. R.G. R.G. Longoria Longoria Dynamic Systems and Controls Laboratory
Department of Mechanical Engineering The University Univers ity of Texas Texas at Austin
Generating Intensity Histogram
Light Meter
Within the the ROI, a histogram is generated of the intensity values. values. Note that most of the image image is made up of pixels with intensity greater than about 180. White is 255. ME 144L – Prof. R.G. R.G. Longoria Longoria Dynamic Systems and Controls Laboratory
Department of Mechanical Engineering The University Univers ity of Texas Texas at Austin
Capturing Images • PCI cards (for capturing from streaming source) • USB USB cam camer eras as* * (w (web cams cams,, et etc.) c.) • Firewire ca cameras • Ethe Ethern rnet et/w /wir irel eles esss came camerras
IMAQdx refers to VIs that can be used with cameras that interface directly (‘direct show’)
• USB USB is the the appr approac oach h tar target geted ed for this this cours course: e: – Low-cost – Rela Relati tive vely ly eas easy y to use use ME 144L – Prof. R.G. R.G. Longoria Longoria Dynamic Systems and Controls Laboratory
Department of Mechanical Engineering The University Univers ity of Texas Texas at Austin
USB Cameras (Webcams) • USB USB webc webcam amss are are prob probabl ably y the the slowe slowest st came camera rass available, especially the way they are to be used in this course. • Our Our expe experi rien ence ce has has sho shown wn tha thatt the the max maxim imum um bandwidth we can achieve for image acquisition is about 10 frames/sec (within LabVIEW). • Some Some onli online ne sourc sources es indi indica cate te that that ‘hac ‘hacked ked’’ webc webcam amss can achieve 30 frames/sec. • So, So, it is the softw softwar aree envi enviro ronm nmen entt (Wi (Windo ndows ws,, LV, LV, communications, etc.) that we’ve chosen that is placing the restrictions on the performance. ME 144L – Prof. R.G. R.G. Longoria Longoria Dynamic Systems and Controls Laboratory
Department of Mechanical Engineering The University Univers ity of Texas Texas at Austin
Acquire a USB Camera Image
ME 144L – Prof. R.G. R.G. Longoria Longoria Dynamic Systems and Controls Laboratory
Department of Mechanical Engineering The University Univers ity of Texas Texas at Austin
Front Panel
Cursor location
NxM, 32-bit RGB image ME 144L – Prof. R.G. R.G. Longoria Longoria Dynamic Systems and Controls Laboratory
RGB levels… Department of Mechanical Engineering The University Univers ity of Texas Texas at Austin
Summary • Vision Vision VIs in LabVIE LabVIEW W provid providee a way for for us us to includ includee image image acquisition and analysis to our existing set of tools (simulation, DAQ). • The The visio vision n VIs VIs alone alone allo allow w you you to use use an an imag imagee as a data data type type.. • This This imag imagee can can be load loaded ed fro from m a file file or or acqu acquire ired d usin using g IMAQ IMAQ routines. • Once Once within within LabVIE LabVIEW, W, the the image image can be proces processed sed using using some some very sophisticated built-in programs. • The labora laborator tory y work work will will introdu introduce ce specif specific ic metho methods ds that that will enable you to build a simple ‘motion capture’ system.
ME 144L – Prof. R.G. R.G. Longoria Longoria Dynamic Systems and Controls Laboratory
Department of Mechanical Engineering The University Univers ity of Texas Texas at Austin
Image Data Type Menu: NI Measurements->Vision->Vision Utilities->Image Management
This VI is used to create an image. It is called called prior to, say, capturing an image using a camera.
ME 144L – Prof. R.G. R.G. Longoria Longoria Dynamic Systems and Controls Laboratory
Department of Mechanical Engineering The University Univers ity of Texas Texas at Austin
Image Type and Bit Depth • • • •
•
•
We know know digita digitall ima images ges are formed formed by an an array array of pixe pixels, ls, and each each pixe pixell is is quantized into a number of levels based on the number of bits available. Depend Depending ing on whet whether her pixels pixels are black black and and whit white, e, gray graysca scale, le, or colo color, r, pixe pixels ls have different bit depths. Bit depth refers to the amount of information allocated to each pixel. When When pixe pixels ls are are eit eithe herr blac black k or whi white te,, pixe pixels ls nee need d only only two two bits bits of of information (black or white), and hence the pixel depth is 2. For graysca grayscale, le, the number number of leve levels ls used used can vary vary but but most most syst system emss have have 256 256 shades of gray, 0 being black and 255 being white. When W hen there are 256 shades of grey, each pixels has a bit depth of 8 bits (one byte). byte). A 1024 x 1024 grayscale images would occupy 1MB of memory. In digi digital tal color color ima images ges,, the RGB (red (red green green blue, blue, for for scree screen n projec projectio tion) n) or CMYK (printing color) schemes are used. Each color occupies 8 bits (one byte), ranging in value from 1-256. Hence in RGB each pixel occupies 8x3 =24 (3 bytes) bits, in CMYK 8x4 = 32 bits (4 bytes). Note, LV uses an ‘alpha’ channel for RGB. The alpha channel stores transparency information--the higher the value, the more opaque that pixel is.
ME 144L – Prof. R.G. R.G. Longoria Longoria Dynamic Systems and Controls Laboratory
Department of Mechanical Engineering The University Univers ity of Texas Texas at Austin
