Crio Projects - Stack Overflow Auto-Search Tool | Crio.Do | Project-Based Learning Platform for Developers

Objective

You will be creating a tool that will execute a Python program and check for errors during execution. If there are errors, the tool will search for Stack Overflow threads that relate to the errors and open them as tabs in a web browser.


Project Context

Stack Overflow is a source of helpful information for software development related queries. A few years post its inauguration, Stack Overflow became one of the norms of software development and is now a fundamental component of software development culture.


While coding, all of us get entangled in one or the other type of error very often. That’s where the aid of sites like Stack Overflow comes into picture. The general pattern of our everyday software development life involves coding and testing the code. In case we encounter errors, we search for those errors on Google, Stack Overflow, etc., fix them and continue this cycle till we get our code running the way we need.


In this project we are going to create a tool (Python script) that will check our Python program for errors by executing it and on encountering any errors, it will search for it on Stack Overflow and open up a few threads in web browser tabs.


Flow Chart

The following flowchart summarizes the project:

image alt text

High-Level Approach

  • Create a wrapper Python script, that will run the test Python code, to check for errors. It’ll simply print "No errors found", in case there are no syntactic or runtime errors.

  • On encountering an error, it’ll extract the required error type and the error message.

  • Then we need to make a REST API call to Stack Exchange API.

  • From the JSON response obtained, we need to extract at most N number of links to Stack Overflow threads and open the same in a web browser.

Pre-requisite skills

  • Python

Post Project Skills

  • REST

  • HTTP

Credits

"Aditya Yogish Pai for project idea"


Objective

You will be creating a tool that will execute a Python program and check for errors during execution. If there are errors, the tool will search for Stack Overflow threads that relate to the errors and open them as tabs in a web browser.


Project Context

Stack Overflow is a source of helpful information for software development related queries. A few years post its inauguration, Stack Overflow became one of the norms of software development and is now a fundamental component of software development culture.


While coding, all of us get entangled in one or the other type of error very often. That’s where the aid of sites like Stack Overflow comes into picture. The general pattern of our everyday software development life involves coding and testing the code. In case we encounter errors, we search for those errors on Google, Stack Overflow, etc., fix them and continue this cycle till we get our code running the way we need.


In this project we are going to create a tool (Python script) that will check our Python program for errors by executing it and on encountering any errors, it will search for it on Stack Overflow and open up a few threads in web browser tabs.


Flow Chart

The following flowchart summarizes the project:

image alt text

High-Level Approach

  • Create a wrapper Python script, that will run the test Python code, to check for errors. It’ll simply print "No errors found", in case there are no syntactic or runtime errors.

  • On encountering an error, it’ll extract the required error type and the error message.

  • Then we need to make a REST API call to Stack Exchange API.

  • From the JSON response obtained, we need to extract at most N number of links to Stack Overflow threads and open the same in a web browser.

Pre-requisite skills

  • Python

Post Project Skills

  • REST

  • HTTP

Credits

"Aditya Yogish Pai for project idea"


Proof of Concept

First validate the idea by doing a low level implementation (Proof of Concept) of the components involved in the project.


This helps you to:

  1. Get more clarity around the unknowns.

  2. Get a better understanding of the stages involved in the project.


[Note: Before proceeding you should go through the REST and HTTP bytes on the platform. Doing so will make you better equipped.]


Requirements

  • Write a one-liner Python program which will give us a runtime error. For example, let’s say we have a program sample.py which contains only one statement a = 5 + 'string'. On running the program, we get a runtime error as shown in the image below. As you can see that the last line contains the error type as TypeError and the error message as unsupported operand type(s) for +: 'int' and 'str'. This is due to the fact that we cannot use the + operator with a numerical value and a string as operands in Python. image alt text

  • Visit api.stackexchange.com and examine the documentation for the usage of the basic search feature.

  • Make a REST API call using the Try It section provided on the website. Use the error message and the programming language (Python in this case) as the values for intitle and tagged parameters respectively.

  • You’ll receive a JSON response which will contain a list of items. Each object in the items list provides us the details of a Stack Overflow thread which matches the required query. We are interested in those objects which have the is_answered parameter set to true. These objects contain a link parameter which contains the link to the respective Stack Overflow thread. You need to open a few links to verify.


References


Bring it On!

  • Try playing around with the Try it section of the basic search feature by setting the values for parameters like fromdate, todate, sort, etc. This will give you a better understanding of how to use the API more efficiently.

  • Can you figure out how to use the advanced search that the Stack Exchange API offers?


Expected Outcome

The main purpose of this milestone is for you to get an idea on how to efficiently use Stack Exchange API to get the required data from Stack Overflow.


Extract the error message by executing the program

In this milestone, you’ll be writing the wrapper Python script to execute a Python test program and check for errors when it’s executed. If it contains errors, you need to filter out the error type and error message from the Traceback. Otherwise, print No errors found on the console.


Requirements

  • Utilize the Popen method from the subprocess module in your wrapper script to execute the test program. It’ll be easier to extract the required error details on doing so.

  • From the previous step, we’ll receive the error details in the form of a binary object. You need to decode it to UTF-8. This will provide the data to us in the form of a string.

  • The data will contain details like file name, line number, wrong statement, the type of error, the error message, etc. We only need the type of error and the error message, hence we need to filter for the same.


References


Expected Outcome

You should be able to come up with a Python script that executes your test Python program. In case errors pop up, it should be able to extract the error type and the error message.


Auto-searching for Stack Overflow threads

After successful extraction of the error message, the next step is to use the Stack Exchange API’s search feature and open up a few Stack Overflow threads that relate to the error.


[Note: Refer to milestone 1 for details on how to use the API.]


Requirements

  • Use the error type and error message extracted and make a GET request using the Stack Exchange API’s search feature, from your wrapper script. You’ll receive a JSON response. Make the following type of queries:

    • Using the error type

    • Using the error message

    • Using the error type and error message combined

  • Extract at most N (5 for example) number of links that relate to Stack Overflow threads which are answered. [Note: Refer to milestone 1 for details on how to access the required links.]

  • Use the webbrowser module to open the extracted links in a web browser.


References


Expected Outcome

By the end of this milestone, the core implementation of the project should be in place. You should have a wrapper Python script that executes a test Python program and checks for errors in it. If errors are seen, it opens up related Stack Overflow threads in a web browser.


Spice it up!

On completion of the earlier milestones, you’ll have a tool that can help you in case you encounter errors in your Python code. But, won’t it be great if you could have a similar tool for languages like Java, C++, etc.? We’ll try to accomplish the same in this milestone.


[Note: This is not a mandatory milestone.]


Requirements

  • Let’s take Java for example. Java programs are both compiled and interpreted. Your script needs to compile a Java program and in case it encounters errors in compilation, it needs to extract the error details from the same.

  • Once you’ve extracted the error message, the rest of the code for the wrapper script will be the same. Modular code helps in code reusability. Make sure that your script’s code for various components like extracting the error, making the API call, extracting the list of links of threads, etc. are in separate methods.


Bring it On!

  • Can you check for errors in the code of other programming languages like C, C++, Go, etc.?

Expected Outcome

You should be able to add the feature of searching for errors in a Java program to your wrapper Python script. Adding more languages is a definite plus!


Publish to GitHub

Awesome job, my fellow coder!


You shouldn’t wait anymore to showcase your work on GitHub, so kindly publish the same.


[Note: Kindly go through this Byte if you’re unfamiliar with Git.]


References