AUTOMATA

Automata is world's most advanced and only programming assessment that uses machine learning for grading programs. 

THE NEED

There is a huge need of good software engineers across the industry given the ubiquity of IT, cloud and mobile based solutions. The industry looks for software engineers who can not only think of the right algorithm for a problem, but also figure out efficient solutions for a problem and write code that is scalable and maintainable. Most current solutions simply find the percent of test cases a program gets right, and not grade based on the thought process of the candidate or quality parameters of a code, as an expert interviewer would have. Also, the talent available at entry-level is very scarce.

Our Solution: AUTOMATA The tool

Automata is the world's smartest coding test to find the most appropriate software engineer for a position. Here the candidate is provided with a couple of programming problems, for which s/he writes the solution in a simulated compiler integrated environment. The candidate can compile and execute his/her code to find how it is performing on test cases.

Grading

To grade a program, Automata goes beyond the simplistic test case criterion using its patent pending technology. Based on semantic analysis of the program and use of machine learning, Automata grades a program on how close a program’s implementation logic is to a possible correct solution. Our machine learning algorithm score matches an expert score by a Pearson correlation of coefficient of 0.85 on average (where test cased based score just matches 0.6) – this means our automated engine grades as well as an expert interviewer would do. It can detect programs which are low on test case score, but are semantically very similar to a correct implementation, thus capturing the thought process of the candidate. In a case study at a recruitment event of a company, it was found that the machine learning score was able to predict 22.6% good candidates, which would have been missed by using just a test case metric.

Automata also intelligently structures the test case scores in a taxonomy: basic, advanced and edge. Rather than a monolithic test case score, Automata tells the interviewer, what percent of basic, advanced and edge cases the program of the candidate misses. This is insightful, for instance, a candidate who just misses edge cases, has got the primary logic of the program correctly. On the other hand, a candidate with the same total test case score may miss some advanced cases, which implies the primary logic is faulty.

Other than this, Automata finds how efficient is the solution in terms of time complexity and code maintainability. For the prior, our algorithm detects time complexity correctly 95% of the times. Within two programs that are logically correct, this measure helps us know which will run faster and is the right algorithm to use in a production environment. Finally, based on some 120+ rules, we detect whether the program is written in a scalable way: is readable by others, not prone to bugs and easily extensible. Graded on a 4-point scale, it distinguishes programs which are readable and extensible, from those which have one or none of these qualities. Given that most time in the IT industry is spent in re-writing and extending legacy code, this feature makes Automata extremely powerful. Finally, Automata also provides a feature to compare the code similarity to other submissions in an event to find if a candidate may have used unfair means to complete the test.

In summary, Automata checks the code logic, its performance on a taxonomy of test cases, its time complexity and code maintainability. Grading on these metrics makes it the most advanced tool out there.

Industry Modules

Automata programming tests come in different versions for hiring software engineers for IT services and IT product companies, entry-level and laterals. A curated question bank, validated empirically, with problems testing different aspects of data structure usage and algorithmic approaches is used on the backend. Automata is available in C, C++, Java and Python. It can be combined with our technology specific modules such as those in Websphere, Power Builder and Java Swings, to form a powerful suite to test candidates for different positions.

Automata Fix

A very high proportion of people in the services industry and also some in the product industry spend their time in debugging existing code and adding new features to it. Automata-Fix addresses finding the right software engineers to do this efficiently. The test provides a set of programs to the candidate, where they have to either debug them or right new functionality based on the given functions and code. It tests the candidate’s skill of being able to understand code written by someone else, be able to find bugs and also use the code to implement new functionality.

The module is also very useful as a screening tool to find candidates who have basic exposure to programming and can be trained in a reasonable time to become software engineers who could write code from scratch.

The Advantage

The real programming skill is not how well you know concepts & constructs but how well you can apply them. Which is why, Automata, our path breaking programming test not only evaluates correctness but also code quality & efficiency in execution. It also provides an objective norm based score for each test. And if that’s not all, the complete test happens in a simulated environment to evaluate the speed and real-time error-handling skills of the candidate.

Automata comes with a scientifically designed question bank consisting of a wide range of easy to hard programming problems with smartly designed corner and generic tests cases. Automata evaluates a candidate's knowledge of algorithms, data structures, concepts like recursion, dynamic memory allocation and modular programming. The compiler supports multiple programming languages like C, C++, Java and more.

Features

• With a variety of basic, advanced and edge test cases designed for that problem, the code is checked thoroughly for correctness and completeness.
• Automata is capable of evaluating the time complexity or simply the efficiency of the code.
• The code is scored on quality of the code based on industry defined best practice.
• A detailed report with objective and subjective evaluation of the code.
• Different versions of Automata available to IT Products and IT Services companies.
• The compiler supports multiple programming languages support like Java, C, C++ and more.

Delivery

Automata can be delivered alongside AMCAT, Aspiring Minds' standardized online, adaptive assessment test. It can be delivered on campuses by setting up a local area network and can also be conducted online through an internet connection. Different versions are available for IT Product and IT services companies looking to select talented programmers. Detailed automated reporting on real time and features like interviewer feedback make it easy to use in a mass scale.

ROI

Automata has shown to be a strong selection criteria in the recruitment process of various companies. It can not only increase recruitment efficiency by up to 65%, but when used as a selection criteria, helps eliminate bad hires thus driving up organizational efficiency.

Need more information? Fill in your details and we will get back to you.