Abstract: Participate in this workshop to learn how to build custom machine classifiers for sifting social media data. The topics covered include how to:
- construct precise social data fetch queries,
- use Boolean search on resulting archives,
- filter on metadata or other project attributes,
- count and set aside duplicates, cluster near-duplicates,
- crowd source human coding,
- measure inter-rater reliability,
- adjudicate coder disagreements, and
- build high quality word sense and topic disambiguation engines.
All workshop participants will get gratis access to DiscoverText for the remainder of 2016. DiscoverText is designed specifically for collecting and cleaning up messy Twitter data streams. Use basic research measurement tools to improve human and machine performance classifying Twitter data over time. The workshop covers how to reach and substantiate inferences using a theoretical and applied model informed by a decade of interdisciplinary, National Science Foundation-funded research into the text classification problem.
Participants will learn how to apply “CoderRank” in machine-learning. Just as Google said not all web pages are created equal, links on some pages rank higher than others, Dr. Shulman argues not all human coders are created equal; the accuracy of observations by some coders on any task invariably rank higher than others. The major idea of the workshop is that when training machines for text analysis, greater reliance should be placed on the input of those humans most likely to create a valid observation. Texifter proposed a unique way to recursively validate, measure, and rank humans on trust and knowledge vectors, and called it CoderRank.
Pre-Workshop Prep:
No Prerequisites Required
Workshop Contact
Stu Schulman – [email protected]
Presenter’s Bio:
Dr. Stuart W. Shulman is founder & CEO of Texifter. He was a Research Associate Professor of Political Science at the University of Massachusetts Amherst and the founding Director of the Qualitative Data Analysis Program (QDAP) at the University of Pittsburgh and at UMass Amherst. Dr. Shulman is Editor Emeritus of the Journal of Information Technology & Politics, the official journal of Information Technology & Politics section of the American Political Science Association.