Venue Information

Venue Bloomberg, LP
Date August 24, Sunday.
Time* 1 PM - 6 PM
Register You would need to register with KDD to participate in this workshop. Please register yourself by clicking here. There is no registration fee.

* Please note: Other workshops will start at 2 PM. We are starting an hour early to have an awesome session. Please be there on time!


Assessing educational achievement and providing feedback to learners is crucial to emerging course work and labor market-making systems. Automating the assessment and feedback mechanisms on open-ended tasks (e.g., short-answer and essay questions) will allow both learning and recruitment to be opened up to a much larger set of people that currently miss out due to resource constraints. Although automated or semi-automated assessment/feedback remains a nascent field, there have been recent advances drawing on techniques from data mining, knowledge discovery, machine learning, and crowdsourcing for peer grading. Notwithstanding, the technical requirements for accuracy and expressiveness for the many purposes for assessment (some high-stakes and some low-stakes; some formative and some summative) are not well-defined.

In this workshop, we will bring together a diverse group of researchers and industry practitioners in data mining, machine learning, and psychology to (1) discuss current state of the art in automated assessment, (2) identify a vision for future research, and (3) lay out a vision for the steps required to build a good automated or peer-grading based assessment. The organizers hope that a unified framework for thinking about assessment and feedback emerges by the end of workshop.

Call for papers

We invite submission of papers describing innovative research on all aspects of assessment of educational achievement and providing feedback. Position papers and papers inducing discussion are strongly encouraged, but should base themselves on fresh insight and/or new data. Topics of interest include, but are not limited to:

  • Automated and semi-automated assessments of open-responses
  • Assessment by crowdsourcing and peer-grading
  • Automatic feedback generation
  • Automatic problem (item) generation/design, calibration, modeling
  • Assessment design, test blue-print, rubric design, validity and reliability
  • Test integrity, equity, proctoring and high-stake testing
  • Non-cognitive assessments in personality, behavior, motivation, etc.
  • New areas: gamification, simulation based assessments
  • Insights derived by large scale assessment data towards learning, recruitment, performance prediction, etc.

Papers submitted should be original work and different from papers that have been previously published. Submission to the workshop does not preclude submission to other venues in future or in parallel. The authors may publish the work submitted here or its extensions at other avenues.
Accepted papers shall be provided a speaking slot at the workshop.

Submission Guidelines

Papers should be no more than 8 pages total in length. Use the standard double-column ACM Proceedings Style format. Additional information about formatting and style files are available online at:

Reviewing will be double-blind : reviewers will not know the identities of the authors.

Paper submissions should be made to ASSESS 2014's EasyChair portal
[Submission deadline extented to 24th June]

Important Dates

Paper submission deadline June 24th, 2014, 23:59 Pacific Standard Time
June 15th, 2014, 23:59 Pacific Standard Time
Acceptance notification July 05, 2014
Final paper due July 15, 2014
Workshop August 24, 2014




Program Schedule

Time Event Description
1 PM - 1:10 PM Introduction 10 minutes, organizers
1:10 PM - 1:50 PM Strapping jet engines to the stage coach: Using technology to push the boundaries of educational measurement Damian Bebell*, Boston College (Education Psychology perspective)
1:50 PM - 2:30 PM Problem Generation & Feedback Generation Sumit Gulwani*, Microsoft Research (CS/ML perspective)
2:30 PM - 3:15 PM 3 Papers 15 minutes each (Presentation Details)

3:15 PM - 3:30 PM

Coffee Break

3:30 PM - 4:25 PM Industry Panel Colin Fuller*, Khan Academy
Piotr Mitros*, edX
Megan Shaw*, Amazon
Moderated by : Varun Aggarwal, Aspiring Minds
4:25 PM - 4:55 PM Posters
4:55 PM - 5:40 PM 3 Papers 15 minutes each (Presentation Details)

5:40 PM - 6:00 PM

Concluding session

Conducted by the organizers to summarize and set out a basic formulation for a white paper

6:00 PM End

* Tentative confirmation received. More speakers and panelists to be finalized


Damian Bebell

Dr. Damian Bebell is an Assistant Research Professor at Boston College's Lynch School of Education and a Senior Research Associate at the Center for the Study of Testing, Evaluation, and Educational Policy. Over the past decade, Damian has led numerous research and evaluation studies investigating the effects of 1-to-1 technology programs and other computer-based technology tools on teaching and learning across wide range of educational settings. Currently, Damian is leading one of the world's first formal evaluation studies of Massive Open Online Courses (MOOCs) with partners MIT and edX studying the efficacy of MOOC models in traditional and non-traditional settings. Damian also currently serves as the Research Director for the International Research Collaborative where he conducts longitudinal research and evaluation studies with international school partners.[PPT]

Sumit Gulwani

Sumit Gulwani is a principal researcher at Microsoft Research, and an adjunct faculty in the Computer Science Department at IIT Kanpur. He has expertise in formal methods and automated program analysis and synthesis techniques. As part of his vision to empower masses, he has recently focused on cross-disciplinary areas of automating end-user programming (for systems like spreadsheets, smartphones, and robots), and building intelligent tutoring systems (for various subject domains including programming, math, logic, and automata). Sumit's programming-by-example work led to the famous Flash Fill feature of Microsoft Excel 2013 that is used by hundreds of millions of people. Sumit is a recipient of the ACM SIGPLAN Robin Milner Young Researcher Award. Sumit obtained his PhD in Computer Science from UC-Berkeley in 2005, and was awarded the ACM SIGPLAN Outstanding Doctoral Dissertation Award. He obtained his BTech in Computer Science and Engineering from IIT Kanpur in 2000, and was awarded the President's Gold Medal.[PPT]


Colin Fuller

Colin is on the Data Science team at Khan Academy, where he helps students get the most out of the time they spend on the site and writes software to help everyone at Khan Academy get the most out of their data. Most recently, he's been working on the team overhauling how Khan Academy determines what users already know and what they're learning outside of the site. Prior to working at Khan Academy, Colin was a postdoctoral fellow and a graduate student in biochemistry at Stanford University, where he developed new image processing and quantitative microscopy methods to understand how cells pass on their DNA.

Piotr Mitros

As chief scientist at edX, Piotr Mitros is charged with developing and applying technology to optimize the learning process. His teaching experience around the world has given him a unique perspective on education. As a graduate student, Piotr took breaks from his thesis to spend time teaching in China, working in India, and facilitating educational technology projects in Nigeria. His observations of university systems in the developing world inspired Piotr to find ways to leverage peer teaching and distance learning to dramatically increase access to education. Following a stint in industry as an analog designer, most recently at start-up Rhythmia Medical where he designed the analog front end for a novel medical imaging modality, Piotr returned to MIT to help lead the creation of the original MITx platform. He brings a broad interdisciplinary background that combines teaching, engineering, computer science, and math, and has been interested in teaching and education since he was a child. Piotr enjoys making things -- curtains, bicycle parts, electronics, furniture, and speakers. Piotr holds a B.S. in math and electrical engineering, a M.S. in EECS, and a Ph.D. in EECS, all from MIT.


Program Committee


7 September - Presentations of the paper presenters and invited speakers uploaded.

24 August - Assess 2014 is today.Great turn out!

31 July - Accepted Submissions (papers/posters) updated.

21 July - List of Panelists updated.

14 July - Venue details updated.

8 July - Paper decisions announced.

19 June - Paper submission deadline extended to 24th June.

14 June - Paper submission deadline extended to 20th June.

20 May - Information on the data contest has been updated.

6 May - Program Committee list updated

21 April - Call for papers announced. June 15 submission deadline.

17 April - Workshop accepted at KDD 2014!