Learning Goal

Have the fundamental understanding of Python Object-Oriented Programming

Apache Spark Tutorials from Udemy

Learning Gap and Lessons Learned

For a while, the python script did not run in the terminal. It ran fine in the interactive window, but I wanted it to run in the terminal as I saw in the tutorial. I tinkered with the environment variables and got a lot more confident setting them up. The issue was not the environment variables, however, it was the way the script was structured, namely, the absence of the following clause.

I found out about this by googling and typing this issue on Youtube. The resources that I referred to are as follows:

https://stackoverflow.com/questions/50510291/python-code-doesnt-execute-no-errors-showed

https://stackoverflow.com/questions/45954528/pip-is-configured-with-locations-that-require-tls-ssl-however-the-ssl-module-in

Notice in the link above that the environment variable to the path to

D:\Anaconda3\Library\bin 

Needs to be set as well.

While I had trouble running the above program, I also looked into what is Anaconda and why I should be using it if ever. The Anaconda folder on my computer seemed quite convoluted with the libraries that I do not understand the uses for. The following video was handy in understanding just that. It basically is an all-in-one virtual environment and package manager. This will come handy as I will be working on multiple projects at the same time.

The tutorial also explains that I could have chosen to download miniconda as opposed to the full conda to avoid having a massive list of libraries that I do not use now.