vii // Contents // List of Figures xvi // List of Tables xix // Acknowledgments xxi // About the Companion Website xxiii // Chapter 1 Introduction 1 // 1.0 Introduction to the Book 3 // What This Book is About 3 // Systematically Working with Video Data: Phases I-V 4 // Chapter 1: Systematically Working with Multimodal Data: Introduction 4 Chapter 2: Philosophical and Theoretical Background 5 // Chapter 3: Systematically Working with Multimodal Data Phase I 6 // Chapter 3.0: Phase I: Data Collection 7 // Chapter 3.1 : Systematically Working with Small Data Sets/Data Pieces: Phase I 7 // Chapter 3.2: Systematically Working with Medium-Sized Data Sets: Phase I 8 Chapter 3.3: Systematically Working with Large Data Sets: Phase I 9 // Chapter 4: Systematically Working with Video Data Phase II 9 // Chapter 4.0: Phase II: Delineating the Data 10 // Chapter 4.1: Systematically Working with Small Data Sets/Data Pieces: Phase II 10 // Chapter 4.2: Systematically Working with Medium-Sized Data Sets: Phase II 11 Chapter 4.3: Systematically Working with Large Data Sets: Phase II 11 Chapter 5: Systematically Working with Video Data Phase III 12 // Chapter 5.0: Phase HI: Selecting Data Pieces for Micro analysis 12 // Chapter 5.1: Systematically Working with Small Data Sets/Data Pieces: Phase III 13 // Chapter 5.2: Systematically Working with Medium-Sized Data Sets: Phase III 13 // Chapter 5.3: Systematically Working with Large Data Sets: Phase HI 14 Chapter 6: Systematically Working with Video
Data Phase IV IS // Phase IV: Transcribing Data Using Multimodal Transcription Conventions IS // Chapter 7: Systematically Working with Multimodal Data Phase V /6 // Phase V: Using Analytical Tools 16 // Chapter 8: Systematically Working with Multimodal Data 17 // A Quick Guide for Instructors 17 // How to Use This Book 17 // viii Contents // 1.1 Brief Introduction to Multimodal Discourse Analysis 19 // Multimodal Discourse Analysis: Some Other Research 19 // Multimodal Discourse Analysis: Research Underpinning This Book 20 // Mediated Discourse Theory 21 // Philosophy and Theory 22 // Timing and Rhythm 23 // Chapter 2 Background 25 // 2.0 Philosophical and Theoretical Background 27 // Philosophical Underpinnings 28 // Primacy of Perception 29 // Primacy of Embodiment 31 // Theoretical Underpinnings: One Principle, Two Sub-Principles 32 // The Principle of Social Action 32 // The Sub-Principle of Communication 33 // The Sub-Principle of History 34 // Principle and Sub-Principles: A Perfect Fit 35 // Multimodal Mediated Theory: Basic Theoretical Concepts 35 // Embodiment and Perception: Multimodal Mediated Theory 36 // Unit of Analysis: The Mediated Action 36 // Social Actors 37 // Mediational Means/Cultural Tools 37 // The Mediated Action 37 // Non-Human Social Actors 37 // Humans and Animals as Mediational Means 38 // Multiple Mediational Means/Cultural Tools in a Mediated Action 39 // Lower-Level Mediated Action 40 // Lower-Level Mediated Actions: Perception 41 // Lower-Level
Mediated Actions: Embodiment 42 // Higher-Level Mediated Action 43 // Higher-Level Mediated Actions: Perception 44 // Higher-Level Mediated Actions: Embodiment 45 // Frozen Mediated Action 45 // Frozen Mediated Actions: Perception 46 // Frozen Mediated Actions: Embodiment 46 // Practices and Discourses 47 // Practice: Perception Practice: Embodiment Site of Engagement // Site of Engagement: Perception 49 // Site of Engagement: Embodiment 49 // 2.1 Development of Scollon’s Philosophical Thought 50 // Scollon Referenced Vygotsky and Nishida 51 // Scollon and Vygotsky: Some Musings 51 // č č č // Contents ix // Scollon and Nishida: Some Musings S3 // Scollons Theory: Some Musings 57 // Scollon’s Practice: Some Musings 57 // Philosophical and Theoretical Background: Summary 59 // Philosophical and Theoretical Background - Things to Remember 60 // Chapter 3 Systematically Working with Multimodal Data Phase I 61 // 3.0 Phase I: Data Collection 63 // So, Where Do We Begin? 64 // Phase I: Data Collection 64 // Step 1: Identifying a Theme or Topic 64 // Step 2: Developing Research Questions 65 // Step 3: Considerations Before Setting Out on Data Collection: // What Is Data? 66 // Step 4: Considerations Before Setting Out on Data Collection: What Are Naturally Occurring Data? 68 // Step 5: Considerations Before Setting Out on Data Collection: Ethics 69 // Working with Participants: Consent 69 // Working with Participants: Informing Them About Your Findings 71 // Fair Use or Permissions:
YouTube, TV, Movies 72 // Step 6: Video Recording and Camera Placement 72 // Limitations of Video Recording 73 // How Much Data Is Enough? 73 // Recording and Camera Placement 75 // Corpora 76 // A Common Reaction to Being Recorded 77 // Using Readily Available Video as Data: YouTube, TV, and Movies 77 // Step 7: Collecting Video Data, Taking Field Notes and Interviewing Participants 79 // Step 8: Producing a Data Collection Table 79 // Step 9: Identifying the Recording Equipment, the Researcher(s), Video Recorded (Inter)Action and Animals Recorded or Not Recorded 79 // Step 10: Collating and Time Stamping 81 // Collating and Synchronizing Videos 81 // Time Stamping Your Video Files 82 // Phase I: Summary 82 // Phase I: Data Collection - Things to Remember 83 // Research Projects 85 // 3.1 Systematically Working with Small Data Sets/Data Pieces: Phase I Data // Collection 86 // A Step by-Step Guide to Analyze a YouTube Video 86 // Phase I: Data Collection 86 // Step 1: Identifying a Theme or Topic 87 // Step 2: Developing Research Questions 87 // Step 3: Considerations Before Setting Out on Data Collection: // What Is Data? 87 // Step 4: Considerations Before Setting Out on Data Collection: What Are Naturally Occurring Data? 88 // Step 5: Considerations Before Setting Out on Data Collection: Ethics 88 // Step 6: Video Recording and Camera Placement 88 // Step 7: Collecting Video Data, Taking Field Notes and Interviewing Participants 89 // Step 8: Producing a Data Collection Table
90 // Step 9: Identifying the Researcher in Relation to the YouTube Video 91 // Phase I: Summary 91 // 3.2 Systematically Working with Medium-Sized Data Sets: Phase I Data Collection 93 // A Step-by-Step Guide to Analyze Experimental Data 93 // Phase I: Data Collection 93 // Step 1: Identifying a Theme or Topic 94 // Step 2: Developing Research Questions 95 // Step 3: Considerations Before Setting Out on Data Collection: // What Is Data? 95 // Step 4: Considerations Before Setting Out on Data Collection: What Are Naturally Occurring Data? 96 // Step 5: Considerations Before Setting Out on Data Collection: Ethics 96 // Step 6: Video Recording and Camera Placement 97 // Limitations of Video Recording 97 // How Much Data Is Enough? 98 // Camera Angle(s), Number of Cameras, and Camera Placement 99 // The Place of the Researcher 101 // Step 7: Collecting Video Data, Taking Field Notes and Interviewing Participants 101 // Step 8: Producing a Data Collection Table 102 // Step 9: Identifying the Recording Equipment, the Researcher(s), and the Video Recorded (Inter)Action 102 // Step 10: Collating and Time Stamping 103 // Phase I: Summary 104 // 3.3 Systematically Working with Large Data Sets: Phase I Data Collection 106 // A Step-by-Step Guide to Analyze Video Ethnographic Data 106 // Phase I: Data Collection 106 // Step 1: Identifying a Theme or Topic 107 // Step 2: Developing Research Questions 108 // Step 3: Considerations Before Setting Out on Data Collection: // What Is Data? 108
// Step 4: Considerations Before Setting Out on Data Collection: What Are Naturally Occurring Data? 109 // Step 5: Considerations Before Setting Out on Data Collection: Ethics 109 Step 6: Video Recording and Camera Placement 110 // Contents xi // How Much Data Is Enough? 110 // Camera Angle(s), Number of Cameras, and Camera Placement 111 // The Place of the Researcher 111 // Step 7: Collecting Video Data, Taking Field Notes, and Interviewing Participants 112 // Step 8: Producing a Data Collection Table 113 // Step 9: Identifying the Recording Equipment, the Researcher(s), and the Video Recorded (Inter)action 113 // Step 10: Collating and Time Stamping 114 // Phase I: Summary 115 // Chapter 4 Systematically Working with Multimodal Data Phase II 117 // 4.0 Phase II: Delineating the Data 119 // So, Where Do We Begin? 119 // Phase II: Delineating the Data 120 // Step 1: Identifying Your Data Set 121 // Step 2: Understanding the Site of Engagement That Includes the Researcher and the Data Set 123 // Step 3: Identifying a Data Piece 124 // Step 4: Understanding the Site of Engagement That Includes the Researcher and the Data Piece 125 // Step 5: Phrasing a Research Question That Encircles Your Data 126 // Phase II: Summary 127 // Phase II: Delineating Your Data - Things to Remember 127 // 4.1 Systematically Working with Small Data Sets/Data Pieces: Phase II // Delineating the Data 129 // A Step-by-Step Guide to Analyze a YouTube Video 129 // Phase II: Delineating the Data 129 // Step
1 : Identifying Your Data Set 129 // Step 2: Understanding the Site of Engagement That Includes the Researcher and the Data Set 131 // Step 3: Identifying a Data Piece 132 // Step 4: Identifying the Site of Engagement That Embraces the Researcher/ Viewer in Relation to the Data Piece 133 // Step 5: Phrasing a Research Question That Encircles Your Data 134 // Phase II: Summary 135 // 4.2 Systematically Working with Medium-Sized Data Sets: Phase II Delineating // the Data 137 // A Step-by-Step Guide to Analyze Experimental Data 137 // Phase II: Delineating the Data 137 // Step 1: Identifying Your Data Set 138 // Step 2: Understanding the Site of Engagement That Includes the Researcher and the Data Set 142 // xii I Contents // Step 3: Identifying a Data Piece 243 // Step 4: Identifying the Site of Engagement That Embraces the Researcher in Relation to the Data Piece 244 // Step 5: Phrasing a Research Question That EncirlcesYour Data 245 // Phase II: Summary 246 // 4.3 Systematically Working with Large Data Sets: Phase II Delineating the Data 148 A Step-by-Step Guide to Analyzing Video Ethnographic Data 148 // Phase II: Delineating the Data 248 // Step 1: Identifying Your Data Set 249 // Step 2: Understanding the Site of Engagement That Includes the Researcher and the Data Set 252 // Step 3: Identifying a Data Piece 254 // Step 4: Identifying the Site of Engagement That Embraces You, the Viewer/Researcher and the Data Piece 255 // Step 5: Phrasing a Research Question That Encircles
Your Data 155 // Phase II: Summary 257 // Chapter 5 Systematically Working with Multimodal Data: Phase III 259 // 5.0 Phase III: Selecting Data Pieces for Micro Analysis 261 // So, Where Do We Begin? 262 // Phase III: Selecting Data Pieces for Micro Analysis 762 // Step 1: The MEDIATED ACTION: Rephrasing Your General Research Question 263 // Step 2: Demarcating Higher-Level Mediated Actions 164 // Step 3: Developing an Overview of Higher-Level Mediated Actions in Your Data Set 755 // Step 4: Narrowing the Site of Engagement 767 // Site of Engagement as Window for Analysis 767 // Phase III: Summary 268 // Phase HI: Selecting Data Pieces for Micro Analysis - Things to Remember // 5.1 Systematically Working with Small Data Sets/Data Pieces: Phase III // Selecting Data Pieces for Micro Analysis 170 // A Step-by-Step Guide to Analyze a YouTube Video 270 // Phase HI: Selecting Data Pieces for Micro Analysis 270 // Step 1: The MEDIATED ACTION: Rephrasing Your Data Driven // Research Question 271 // Step 2: Demarcating Higher-Level Mediated Actions 272 // Step 3: Developing an Overview of Higher-Level Mediated Actions in Your Data Set 774 // Step 4: Selecting Data Pieces for Micro Analysis: Narrowing the Site of Engagement 7 75 // Phase III: Summary 777 // Contents // 5.2 Systematically Working with Medium-Sized Data Sets: Phase III Selecting Data Pieces for Micro Analysis // A Step-by-Step Guide to Analyze Experimental Data Phase III: Selecting Data Pieces for Micro Analysis Step 1:
The MEDIATED ACTION: Rephrasing Your Data Driven Research Question // Step 2: Demarcating Higher-Level Mediated Actions // Step 3: Developing an Overview of Higher-Level Mediated Actions // in Your Data Set // Step 4: Selecting Data Pieces for Micro Analysis: Narrowing the Site of Engagement Phase III: Summary // 5.3 Systematically Working with Large Data Sets: Phase III Selecting Data Pieces for Micro Analysis // A Step-by-Step Guide to Analyze Video Ethnographic Data Phase HI: Selecting Data Pieces for Micro Analysis Step 1: The MEDIATED ACTION: Rephrasing Your Data Driven Research Question // Step 2: Demarcating Higher-Level Mediated Actions // Step 3: Developing an Overview of Higher-Level Mediated Actions // in Your Data Set // Step 4: Selecting Data Pieces for Micro Analysis - Narrowing the Site of Engagement Phase III: Summary // Chapter 6 Systematically Working with Multimodal Data Phase IV // 6 Phase IV:Transcribing Data Using Multimodal Transcription Conventions So, Where Do We Begin? // Phase IV: Transcribing Data Using Multimodal Transcription Conventions // Step 1 : Identifying a Mode Step 2: Transcribing Layout Step 3: Transcribing Proxemics Step 4: Transcribing Posture Step 5: Transcribing Gesture Step 6: Gaze // Step 7: Transcribing Head Movement Step 8: Transcribing Facial Expression Step 9: Transcribing Object Handling Step 10: Transcribing Language Step 11 : Producing a Final Transcript Phase IV: Summary // Phase IV: Transcribing Data Using Multimodal T ranscription
Conventions - Things to Remember // 178 // 178 // 178 // 179 // 180 // 183 // 185 // 186 // 187 // 187 // 187 // 188 189 // 192 // 194 // 195 // 197 // 199 // 200 // 201 // 201 // 205 // 209 // 211 // 214 // 217 // 219 // 222 // 224 // 225 // 229 // 230 // 230 // xii I Contents // Step 3: Identifying a Data Piece 243 // Step 4: Identifying the Site of Engagement That Embraces the Researcher in Relation to the Data Piece 244 // Step 5: Phrasing a Research Question That EncirlcesYour Data 245 // Phase II: Summary 246 // 4.3 Systematically Working with Large Data Sets: Phase II Delineating the Data 148 A Step-by-Step Guide to Analyzing Video Ethnographic Data 248 // Phase II: Delineating the Data 248 // Step 1: Identifying Your Data Set 249 // Step 2: Understanding the Site of Engagement That Includes the Researcher and the Data Set 252 // Step 3: Identifying a Data Piece 254 // Step 4: Identifying the Site of Engagement That Embraces You, the Viewer/Researcher and the Data Piece /55 // Step 5: Phrasing a Research Question That Encircles Your Data 155 // Phase II: Summary 257 // Chapter 5 Systematically Working with Multimodal Data: Phase III 159 // 5.0 Phase III: Selecting Data Pieces for Micro Analysis 161 // So, Where Do We Begin? 762 // Phase III: Selecting Data Pieces for Micro Analysis 162 // Step 1: The MEDIATED ACTION: Rephrasing Your General Research Question 763 // Step 2: Demarcating Higher-Level Mediated Actions 164 // Step 3: Developing an Overview of Higher-Level Mediated
Actions in Your Data Set 765 // Step 4: Narrowing the Site of Engagement 767 // Site of Engagement as Window for Analysis 767 // Phase HI: Summary 765 // Phase HI: Selecting Data Pieces for Micro Analysis - Things to Remember 763 // 5.1 Systematically Working with Small Data Sets/Data Pieces: Phase III // Selecting Data Pieces for Micro Analysis 170 // A Step-by-Step Guide to Analyze a YouTube Video 170 // Phase III: Selecting Data Pieces for Micro Analysis 170 // Step 1: The MEDIATED ACTION: Rephrasing Your Data Driven Research Question 777 // Step 2: Demarcating Higher-Level Mediated Actions 172 // Step 3: Developing an Overview of Higher-Level Mediated Actions in Your Data Set 774 // Step 4: Selecting Data Pieces for Micro Analysis: Narrowing the Site of Engagement 7 75 // Phase III: Summary 7 77 // Contents // XIII // 5.2 Systematically Working with Medium-Sized Data Sets: Phase III Selecting Data // Pieces for Micro Analysis 178 // A Step-by-Step Guide to Analyze Experimental Data 178 // Phase HI: Selecting Data Pieces for Micro Analysis 178 // Step 1: The MEDIATED ACTION: Rephrasing Your Data Driven Research Question 179 // Step 2: Demarcating Higher-Level Mediated Actions 180 // Step 3: Developing an Overview of Higher-Level Mediated Actions in Your Data Set 183 // Step 4: Selecting Data Pieces for Micro Analysis: Narrowing the Site of Engagement 185 // Phase III: Summary 186 // 5.3 Systematically Working with Large Data Sets: Phase III Selecting Data Pieces // for Micro
Analysis 187 // A Step-by-Step Guide to Analyze Video Ethnographic Data 187 // Phase III: Selecting Data Pieces for Micro Analysis 187 // Step 1 : The MEDIATED ACTION: Rephrasing Your Data Driven Research Question 188 // Step 2: Demarcating Higher-Level Mediated Actions 189 // Step 3: Developing an Overview of Higher-Level Mediated Actions in Your Data Set 192 // Step 4: Selecting Data Pieces for Micro Analysis - Narrowing the Site of Engagement 194 // Phase HI: Summary 195 // Chapter 6 Systematically Working with Multimodal Data Phase IV 197 // 6 Phase IV: Transcribing Data Using Multimodal Transcription Conventions So, Where Do We Begin? // Phase IV: Transcribing Data Using Multimodal Transcription Conventions // Step 1: Identifying a Mode Step 2: Transcribing Layout Step 3: Transcribing Proxemics Step 4: Transcribing Posture Step 5: Transcribing Gesture Step 6: Gaze // Step 7: Transcribing Head Movement Step 8: Transcribing Facial Expression Step 9: Transcribing Object Handling Step 10: Transcribing Language Step 11 : Producing a Final Transcript Phase IV: Summary // Phase IV: Transcribing Data Using Multimodal Transcription Conventions - Things to Remember // 199 // 200 // 201 // 201 // 205 // 209 // 211 // 214 // 217 // 219 // 222 // 224 // 225 // 229 // 230 // 230 // xiv Contents // Chapter 7 Systematically Working with Multimodal Data Phase V 233 // 7 Phase V: Using Analytical Tools 235 // Step 1: Selecting Analytical Tools for Micro Analysis 236 // Analytical Tool for
Micro Analysis 1: The Lower-Level Mediated Äction 236 // Analytical Tool for »Micro Analysis 2: The Higher-Level Mediated Action 238 // Analytical Tool for Micro Analysis 3: The Frozen Mediated Action 241 Analytical Tool for Micro Analysis 4: Modal Density 242 // Analytical Tool for Micro Analysis 5: Modal Configuration 245 // Analytical Tool for Micro Analysis 6: Modal Density Foreground-Background Continuum of Attention/Awareness 247 // Analytical Tool for Micro Analysis 7: Semantic/Pragmatic Means 250 // Step 2: Conducting a Micro Analysis by Utilizing Analytical Tools 253 // Utilizing Analytical Tool for Micro Analysis 1: The Lower-Level Mediated Action 253 // Utilizing Analytical Tool for »Micro Analysis 2: The Higher-Level Mediated Action 254 // Utilizing Analytical Tool for .Micro Analysis 3: The Frozen Mediated Action 254 // Utilizing Analytical Tool for Micro Analysis 4: »Modal Density 255 // Utilizing Analytical Tool for Micro Analysis 5: Modal Configuration 255 // Utilizing Analytical Tool for Micro Analysis 6: Modal Density Foreground-Background Continuum of Attention/Awareness 256 // Utilizing Analytical Tool for Micro Analysis 7: Semantic/Pragmatic Means " 256 // Step 3: Selecting Analytical Fools to Put »Micro Analyses into a Larger Perspective 257 // Analytical Tool for Intermediate/Macro Analyses 1: // Scales of Action 257 // Analytical Tool Set for Intermediate/Macro Analyses 2: Site of Engagement, Practices, and Discourses 253 // Analytical Tool for Intermediate/Macro
Analyses 3: Natural Experienced Time Cycles and Rhythms 260 // Step 4: Putting Micro Analyses into a Larger Perspective by Utilizing Analytical Tools or Tool Sets for Intermediate/Macro Analyses 263 // Utilizing Analytical Tool for Intermediate/Macro Analyses 1: // Scales of Action 263 // Utilizing Analytical Tool Set for Intermediate/Macro Analyses 2: // Site of Engagement, Practices, and Discourses 263 // Utilizing Analytical Tool or Intermediate/Macro Analyses 3: // Natural Experienced Time Cycles and Rhythms 264 // Phase V: Summary 264 // Phase V: Using Analytical Tools - Things to Remember 265 // Contents XV // Chapter 8 Systematically Working with Multimodal Data: // Guides for Instructors 267 // A Quick Guide for Instructors 269 // A Quick Guide for Undergraduate-Class Instructors 270 // A Quick Guide for Graduate-Course Instructors 272 // A Quick Guide for PhD Seminar Instructors and Research Team Leaders 274 // References 277 // Index 296