Introduction Over the course of my working life I have had the opportunity to use many programming concepts and technologies to do countless things. Some of these things involve relatively low-value fruits of my labor, such as automating the error prone or mundane like report generation, task automation, and general data reformatting. Others have been much more valuable, such as developing data products, web applications, and data analysis and processing pipelines. One thing that is notable about nearly all of […]

Introduction When you start a program on your machine it runs in its own “bubble” which is completely separate from other programs that are active at the same time. This “bubble” is called a process, and comprises everything which is needed to manage this program call. For example, this so-called process environment includes the memory pages the process has in use, the file handles this process has opened, both user and group access rights, and its entire command line call, […]

I prefer to work with Python because it is a very flexible programming language, and allows me to interact with the operating system easily. This also includes file system functions. To simply list files in a directory the modules os, subprocess, fnmatch, and pathlib come into play. The following solutions demonstrate how to use these methods effectively. Using os.walk() The os module contains a long list of methods that deal with the filesystem, and the operating system. One of them […]

What is a Circular Dependency? A circular dependency occurs when two or more modules depend on each other. This is due to the fact that each module is defined in terms of the other (See Figure 1). For example: functionA(): functionB() And functionB(): functionA() The code above depicts a fairly obvious circular dependency. functionA() calls functionB(), thus depending on it, and functionB() calls functionA(). This type of circular dependency has some obvious problems, which we’ll describe a bit further in […]