A COMPUTING PRIMER
Current computing technology is not only built upon, but directly mixed in with pieces of its historical evolution. Because there is so much 'legacy cruft' (old code and hardware design) at the heart of virtually every type of computing technology, and because every sort of tech is built from combinations of earlier innovations, it is virtually impossible to learn about one area of computing, without having at least a basic understanding of all the underlying and intersecting fundamental parts. This document provides some basic technical explanation and historical context about how modern PCs, servers and mobile devices, as well as operating systems, software, hardware, the Internet, and other current technologies evolved. The goal is to clarify the foundations of how and why current tech works the way it does, along with what the terminology means. This text should be just enough of an overview to tie together many necessary basic concepts, and to provide a basis for learning practical skills related to modern computing, in greater depth elsewhere.
BITS, BYTES AND THE FOUNDATIONS OF COMPUTING
Computers use billions of tiny 'transistors' (electronic on-off switches) to manipulate 'data' (information), and to control 'hardware' (physical componentry). Early digital computers such as the 'Eniac' used vacuum tubes to represent on-off 'bit' states. By wiring together and collecting the on-off states of thousands of those electronic tube switches, the represented ones and zeros could be used to perform math and logic operations, and to control rotary numerical displays (more on how this fundamental tech works, in a moment). As technology progressed, transistors replaced the on-off (1-0) bit capability of tubes, with faster, smaller, and cheaper components, then 'MOSFET' transistors made it possible to build 'integrated circuits', which fit billions of transistors on a single chip, forming the fundamental building block of modern digital electronics. For many decades, the number of transistors on integrated circuits doubled approximately every 2 years, following 'Moore's Law'. The nanometer manufacturing process required to maintain the progression of Moore's Law is currently reaching its physical limit (parts can't be manufactured any smaller at the atomic level), so instead of putting more transistors on each chip, manufacturers have begun putting more 'core' chips into computers to help process data faster, in 'parallel' (with multiple chips performing calculations simultaneously).
'Binary' notation is at the heart of what makes microscopic one-zero switches useful. Large numbers can be represented by a series of binary 1-0s (on-off bit states). Each column in binary notation represents a power of 2, so 11111111 in binary = 255 in decimal (just add up each column of powers of 2: 128 + 64 + 32 + 16 + 8 + 4 + 2 + 1 = 255). A 'byte' of 8 binary digits (8 columns of ones and zeros) is USEFUL because it can in turn be used to represent other characters from a lookup table. For example, in the 'ASCII' table (American Standard Code for Information Interchange, which is used as the basis of our modern 'Unicode' character set), the character 'A' is represented by the number 65 (01000001 in binary), 'B' is the number 66 (01000010), the '+' symbol is number 43 (00101011), etc. By 'encoding' characters and instructions this way, virtually *any data or operation* can be represented by a series of binary ones and zeros.
'RAM' (Random Access Memory) stores data while the computer is powered on, in MOSFET 'cells', which flip-flop their state, so that high power represents 1, and low power represents 0. The computer's 'CPU' (Central Processing Unit) performs 'fetch', 'decode', and 'execute' instructions upon the bits in memory, to perform useful calculations upon the data. These calculations are performed using 'logic gates', which also rely on flipping 1-0 values. Logic gates act as barriers in electronic circuits which take in and compare two electric currents, and send out a new current based on the result of the comparison. All the computations in a calculator can be handled by logic gates (Googling this topic is a fun way to understand more about how CPU hardware works). The results of calculations performed by the CPU are sent to other components in the system over 'bus' connections on the motherboard. That data, once processed by the CPU, is then put to use by 'microcontrollers' in other parts of the system, which control the actions of specialized hardware components. 'Graphics' controllers, for example, display numbers, characters, and images/video on a display. 'Audio' controllers produce sound through speakers. 'Network' controllers send and receive data over wired and wireless connections. Mechanical controllers produce movements in motorized equipment, for use in robotics and other machines (more about how microcontrollers work in a moment). All these integrated controllers working together use the computations performed upon data by the CPU, to produce functionality which is meaningful and useful to humans: to accept typed characters from a keyboard and to display them on screen, to calculate cells in a spreadsheet, to edit text documents, to play video games, watch movies, view web pages and communicate with others over network connections, to experience immersive virtual reality worlds in VR headsets, to control motors in cars, etc. Modern CPUs process all this useful data in large 'word' chunks of 64 bits at a time ('64 bit architecture').
The 'microcontrollers' in a computer make hardware perform specified operations by turning electricity on and off within circuits, according to whether particular bits (or bytes) in their memory are set to a given number, by input from connected components. An LED light, for example, can be turned on or off by switching power to its circuit, using a simple bit state mechanism in a microcontroller: if a specified bit in its RAM memory is set to 1, the microcontroller turns power to the LED on. If the bit state of that memory location is set to 0, the microcontroller turns the LED's electricity off (via a physical 'pin' connection). To make the LED's brightness increase or decrease, for example, the amount of power sent to the LED can also be controlled by on-off states of other memory cells in the microcontroller. The instructions for how a microcontroller should interpret RAM data, is stored in ROM (Read Only Memory). *Any other device* can potentially be engineered so all of it's functionality is controllable in a similar way. Making a pixel on a screen display a colored dot, sending a specific voltage to a buzzer to make it vibrate at a given frequency, sending the keycode of a letter typed on a keyboard, storing a character to a memory location on a magnetic drive, etc. - all those components of a computer work according to the same basic principal: bit/byte states control electrical voltages sent to hardware, and hardware responds as designed, to those voltages. In this way, microcontrollers make the useful hardware inside a computer, and peripherals attached to it externally, perform the operations which they were designed to perform, according to how the data (collections of bit states) that get sent to them, have been processed by the CPU. All those simple little bits, and the ways they can be calculated by the CPU, accumulated and moved around in RAM, and switched on/off within microcontrollers in a system, etc., are the basis for how every computer achieves such complex (sometimes apparently magical) functionality.
The bytes that make up 'programs' (defined/coded instruction routines created by human users) to orchestrate how the components in a computer work together, and any data (information input/output processed by interaction with those programs), all gets saved to a 'hard drive' (permanent magnetic memory storage device), so that they're not lost when a computer is turned off.
If you're interested in learning more about how the complex electronic architecture of each part in a computer works, use Google. A search about 'how microcontrollers work internally', for example, will yield deep dives into how analog/digital components, UART I/O, RISC/SISC instruction sets, and Interrupt schemes operate at the 'metal' level.
Working directly with the metal componentry inside a computer, by feeding CPUs and microcontrollers 'machine language' ones and zeros, is a very complex undertaking. Luckily, computer systems are designed to hide that complexity from humans, by providing layers of 'higher level' interface technology which make using the hardware more natural to interact with - so that we humans don't need to think at all about the ones and zeros, or voltages sent by microcontrollers to attached hardware, or binary encoded data, etc. A fundamentally important layer in that mix is the 'operating system'.
OPERATING SYSTEMS
A computer's 'Operating System' (OS) is 'software' (a collection of coded programs) that controls all the features of the system hardware. Windows, Mac, and Linux are common modern desktop OSes. Android and iOS are common modern mobile OSes. Operating systems are loaded when the computer starts, by a small program called the 'BIOS' (basic input-output system), which is built into the permanent 'CMOS' memory of a computer. BIOSes provide very basic access to the microcontrollers for screen, keyboard/mouse, hard drive, network, and other fundamental hardware which make up the computer (i.e., they make the screen and keyboard work when the computer turns on). The BIOS remains permanently a part of the computer hardware - it's purpose is to hand control over to whatever OS software is currently prepared to be loaded from a data storage device (magnetic Hard Drive, optical CD drive, USB Flash drive, etc.), when the computer boots up.
Because the operating system consists of software code, it can be upgraded, or even replaced completely by a different operating system. For example, a computer which comes from the factory with Microsoft Windows installed, can be wiped clean and a user can install Linux instead (or any other OS that supports the hardware architecture of that machine). The OS is a very complex suite of software packages which provide complete control of all the hardware built into, and eventually attached to a computer. It provides a naturally usable human interface to that hardware, and provides a 'high level' (human usable) platform upon which application software can be written, and then accessed by users. Most modern operating systems update themselves by downloading software from the Internet, whenever improvements are made to the system design.
'BASIC' MACHINES
Early desktop computers such as the Tandy TRS-80, Commodore 64, Apple II, and TI99-4a didn't have complex operating systems that were visible to users, as on modern PCs. Instead, they typically booted directly into a 'BASIC' programming language environment, which provided simple access to numerical calculations, text string manipulations, graphics, sound and other microcontroller operations, etc., so that users never needed to work with the technical operating system or machine language guts of the computer - that was all hidden beneath, and controlled by, the BASIC interface. BASIC allowed users to structure and save simple code commands on numbered lines, to create software programs ('applications', the old term for 'apps'), to be operated by users of the machine. It was a popular practice for users of these early hobby computers to type in program code found in printed magazines, to be able to play simple games, to control hardware peripherals attached to the computer, or to provide some useful functionality such as text editing (simple word processing), sound and graphic demos, etc. BASIC allowed hobby software creators to assign 'variable' labels to text and numbers input by users of a program, to perform mathematical calculations, 'conditional' evaluations (if-then decisions), 'gosub-return' commands to jump to 'subroutine' sections of code based on a given state, 'loops' to repeat sections of code, and built-in 'function' commands (words which represented actions) to control graphics on screen, sounds produced by a built-in speaker, the saving/loading of program files, etc. BASIC programs were typically saved to a cassette tape, small removable 'floppy disks', or a larger permanent 'hard drive' (HD) attached to the computer.
On some BASIC computers, in order to create more complex programs, machine language could be placed inline within BASIC code to provide extended capabilities not available in BASIC. Using machine language involved sending very detailed and archaic control codes (bits and bytes) to the hardware in the computer. This usually required much more verbose programming to perform even the most basic tasks, typically by moving around memory and controlling the functionality of the CPU very specifically, and placing numbers in memory registers within microcontrollers built into the computer. Machine language required deep technical knowledge of the fundamental architecture of the system, and laborious work to create any sort of useful app, but provided very fast performance and full control of the machinery in the computer. While BASIC was meant to be easy to use, and to provide lots of simply useful commands, it ran slowly, as it had to be 'interpreted' into machine language, while the code was running (this demanded much of the CPU's computing power, just interpreting the language).
Commercial applications for these early systems, typically written in machine language by software development companies, were most often delivered in the form of physical ROM (read-only memory) cartridges which plugged into the back of the computer, and which ran automatically when the computer was turned on (instead of booting to the BASIC programming environment for the user). This cartridge setup allowed non-technical users to run powerful programs without needing any understanding whatsoever of how the computer worked internally. Portable game systems such as the Gameboy, Nintendo DS, and the modern Switch have used a similar cartridge design.
DOS AND COMMAND LINE OPERATING SYSTEMS
The next generation of computers to become popular was the IBM compatible desktop PC ('personal computer') family, which ran an OS called 'DOS'. This early modern operating system was 'console' (interactive text) based. DOS was the standard OS found on IBM compatible computers during the 1980s. The BIOS would load DOS, and present a 'command line' at which users could type commands. DOS included commands which allowed users to format (wipe) a floppy disk or hard drive, view text files, install appropriate drivers to run attached monitors (screens), keyboard/mouse, network card, and other hardware. It also provided commands to list and view files on the hard drive, and allowed users to run any 'executable applications' ('compiled' machine language programs which typically ended with '.exe' in their file name). 'Batch' files (with names ending in '.bat'), allowed users to save a series of commands in a text file, and then run those commands all at once, with some limited interactivity. In command line OSes, you could type the file name of a program, along with some optional 'parameters', which would make the program operate in a specified way. For example, typing 'help' at the command line would run the help program, which printed out basic information about the programs built into DOS. Typing 'help -dir', for example, would provide specific detailed info about the 'dir' command. To make the most basic use of DOS, you need to know the 'dir' command, to list files/folders on a disk (note that folders are also called 'directories' in DOS), the 'cd' (change directory) command to change between folders, and the 'mkdir' command to make new folders. In DOS, '\.' refers to the current folder, and '\..' refers to the parent of the current folder (the folder which contains the current folder). The basic folder and file system from DOS is still used in Windows machines today. You can access the DOS command line in modern versions of Windows by running 'cmd'.
Other command line OSes included Unix, and later Linux, which evolved as an 'open source' free alternative to Unix (anyone can download, work on, and suggest changes/improvements to the code base of open source software, or 'fork' their own version of the software). The built-in commands in Unix/Linux are different from those in DOS (you have to memorize a different set of commands and program names), but the general work flow is the same: type memorized command names at a console prompt to run programs. In Unix/Linux based OSes, you typically type the name of a command or program, plus '--help', to see its optional parameters. For example 'ls --help' shows you how to use the 'ls' command to list the files and folders on a Unix computer. Be aware that Mac and Android are both Linux/Unix based operating systems, so working with the command line on those systems requires learning Linux/Unix console commands. You can access the console on a Mac by clicking Applications > Utilities > Console.
EARLY 'KILLER' APPLICATIONS, BUS SYSTEMS, AND STANDARDIZATION
The modern computing revolution, especially its use in business, really began with the arrival of 'Word Processor' and 'Spreadsheet' programs that ran in DOS. Word processor applications ('WordStar' and 'WordPerfect' were the first) allowed users to type and edit large text files, in ways that were far more productive than type writers. Spreadsheets ('VisiCalc' and 'Lotus 123' were the first) enabled users to enter rows and columns of data, and then to perform calculations, apply formulas, and execute useful logical manipulations upon those tables of data, in ways which could be applied to all sorts of business situations. Spreadsheets were especially useful in accounting, inventory management, scheduling, and any other business process that involved working with collections of customized data. This made PCs valuable to businesses, and helped lead to the widespread adoption of desktop PCs. To this day, suites of 'office' software, such as MS (Microsoft) Office and Google Docs, have word processors (Word) and spreadsheets (Excel) at their core.
Desktop PCs contained a 'bus' system, which was simply a standardized set of connector slots on the 'motherboard' inside the computer case, which could be used to add new hardware to a computer system. Sound cards, video cards, and specialized cards that drove external peripherals were all originally manufactured to fit into standard ISA (Industry Standard Architecture) 8 and 16 bit bus connectors. This expandable bus architecture gave birth to a thriving industry of add-on hardware which extended the physical capabilities of desktop computers, and this bus design still works basically the same way in modern PCs (gaming systems, servers, etc.), although with newer standard 32 bit PCIe (peripheral component interconnect express) connectors. Hard drives were connected via standard ATA/IDE (and now newer SATA and SCSI) bus connectors.
'Driver' software and user programs, to control how data was sent and received by added hardware pieces, allowed users of a computer to perform interesting and useful operations with the additional hardware. Network and modem cards added important communications capabilities to computers. PCs could be connected by 'coaxial' (and later 'twisted pair' 'cat5' and 'cat6') cables that were attached to network cards installed in the motherboard bus of desktop PCs. Modems allowed computers to dial up and connect with one another over a phone line connection. 'BBS' (Bulletin Board) systems were popular during this period, as they allowed users to dial into a community server computer with their phone modem, to upload and download 'shareware' program files, and to communicate with others via notes stored in text files. Transmission 'protocols' were devised to standardize how these devices connected and transferred information over the physical connections. These protocols would initiate the connection with some designated bytes of 'handshake' data that set the number of bits to send in each back-and-forth network 'packet' transmission, the 'baud rate' (speed) for the modems to synchronize at, what to do when an error in transmission occured, etc. The later evolution of network protocols such as 'TCP/IP' and 'UDP' formed the foundation of how machines connect in a standardized way, to form the 'Internet'.
There was an explosion of computer based audio, visual, and document technologies during the 1990s which made use of the hardware and control systems being standardized at the time. File formats used to represent images (BMP, JPG, GIF, PNG, etc.), sound files (WAV, MP3, etc.), video (MOV, MP4, etc.), documents (PDF, DOC, XLS, etc.), as well as raw data formats (Ascii TXT, CSV and many others) were accepted and used universally to store and transport data between programs made by different vendors, so that every application could make use of data files created within other programs.
ASM AND THE C PROGRAMMING LANGUAGE
Early versions of DOS included a BASIC interpreter program, because BASIC was so well known at the time, but most commercial software was written in 'assembly' code or the 'C' language. Assembly language ('ASM') was basically machine code, with some readable 'mnemonics' to make common operations easier for humans to write and understand when reading existing code (this was much better than working with pure 1s and 0s). Writing in assembly language allowed software developers to control the 'metal' of the machine very specifically, and to maximize the speed of a program's performance, but it was still extremely time consuming to write. The C language was the first popular language to help developers create software that ran at optimized speed, with very close control of the metal, but with far more readable/writable human language syntax and control structures than in ASM. C also enabled 'portable' program code to be written, which meant that it could be 'compiled' into an executable program (able to be loaded and run by typing its name at the command line), on all different operating systems. Instead of writing different ASM code for each various type of computer hardware architecture and each different OS in popular use, software developers could write just a single 'source code' program, and then use a 'compiler' on any machine to turn that code into a program that executed the machine language of that particular machine/OS. This 'cross-platform' ability of C, together with its ability to tightly control the metal hardware of the machine, in a productive human-readable way, led to it becoming the basis of all modern 'system' programming work. Most of the operating systems, big programs such as browsers, and even other higher level (more productive and less verbose) language compilers/interpreters, as well as other fundamentally useful software in modern use is typically written primarily in C.
GUI OPERATING SYSTEMS
During the 1990s, most computers began to use 'windowed' operating system interfaces, or 'GUIs' (graphical user interfaces). GUIs allowed users to click on icons with a mouse, instead of typing commands at a console. The original Macintosh desktop computer, and Microsoft Windows 3.1 were the first most popular windowed operating system GUIs. MS Windows was actually a set of programs which ran on top of DOS, to provide all the same functionality found in DOS, but with a simple point and click interface. The 'file manager' program built into Windows allowed users to click on folders, instead of typing 'CD' plus the name of the folder, and to click on program files to run them, instead of typing their file name at the console. Right-clicking the mouse on a file would provide a pop-up menu of usage parameters which could be selectively applied to a file (delete, rename, copy, etc.). GUI systems also provided built-in 'multi tasking' - the ability to run multiple programs at the same time, and to easily switch between each open application by just minimizing or re-sizing each application's window on screen.
The concept of a 'menu' was important in even the earliest GUI operating systems. A row of words along the top of the screen could be clicked on, and each word would drop down a list of grouped options which could be selected by the user's mouse. Sub-menus of those selections made for a very compact, intuitive, and easy to navigate way of providing lots of functionality to users, without ever requiring them to remember how to type commands and optional cryptic parameters. GUIs made computers much more natural to operate for less technically interested users. Virtually any functionality could be applied to clicking on buttons or icons, or selecting options from menus, and data could be entered into text fields, or selected from dropdown lists, or viewed/edited in grid displays, etc.
Early versions of GUI operating systems also standardized much of how hardware peripherals could be added. Hardware could be plugged into the bus slots inside the case of a desktop PC, or connected via cables to a 'serial' or 'parallel' port on the exterior of the case (or later, to 'USB' ports). 'Driver' software (programs which made the hardware's functionality available to the OS), could be installed just by running a program that was delivered on floppy disk, CD, or some other media (or later downloaded from the Internet). This 'plug and play' functionality was standardized to make expanding PCs a simple non-technical matter, consistent with how all the rest of the OS worked - by just pointing and clicking.
GUI OSes provide an 'API' (Application Programming Interface), which developers use to make software with 'higher-level' functionality. The OS API is made up of a collection of 'functions' in 'libraries' which can be 'called' (executed) in languages such as C. There are functions to place GUI 'widgets' on screen (buttons, text fields, drop-down selectors, menus, etc.) and to draw graphics in a window, to send data through standard protocols over network connections, to read and write data to files, to play sounds, to print documents to a printer, to control peripheral hardware, and to do basically anything else the OS enables. The API of each OS is completely different, which means that different function names and data structures, and even really the fundamental language and way you organize/work with code on each OS is different. So, in the same way that earlier machine language was different for each machine, and C was developed to be portable across varied machines, new programming languages were designed to be portable across operating system APIs.
MORE PROGRAMMING LANGUAGES
Languages such as Pascal, C++, Java, Python, Ruby, C#, variants of Visual Basic, LUA and thousands of others grew out of developers' desire to write code once and run on every operating system. These new languages provided a single consistent set of functions and code structuring features for software developers to use across OSes, but under the hood, converted that code to each OS's different native API code. There also developed among common programming languages a great interest in further simplifying the amount of code needed to achieve common goals, despite the required underlying technology being significantly complex. This allowed deep layers of complexity to be hidden from software developers and users of computers. Most software application developers using modern languages rarely work directly with the ones and zeros being pushed around on the metal of the computer's components, or even with operating system APIs. The exception to this rule tends to be developers of hardware components, who write the software driver interfaces which control metal componentry. They create low-level interfaces to the OS API layer, which is in turn simplified by higher level language layers that developers use to more easily interact with that hardware.
The explosion of programming languages during this period enabled software developers to create apps more quickly and easily. The principles of User Interface ('UI') design - how all those GUI elements could be used to make typical 'front end' windows for apps - become generally standardized. 'Database' systems also became commonplace, typically storing 'tables' (rows and columns) of data with a standardized language called 'Sequel' (SQL), in 'relational' database management systems, or 'DBMS'es.
The combination of hardware, OS, supported software applications, programming languages, and other tooling, is often called a 'platform'. The entire Microsoft Windows and Macintosh families of machines, for example, along with all the software development tools and applications available to run on them, are described as separate 'platforms' for computing.
THE INTERNET
Perhaps the most significant development in computing during the 1990s was the creation of the Internet and the 'World Wide Web'. The Internet was based on basic network technology, tied together by a series of existing protocols, and standardized for connectivity across a web of connected machines. Several key technologies made the Internet immediately easily useful for non-technical people. First was the creation of the 'web browser' and related protocols/languages/technologies. The very first web browsers simply allowed text documents to be displayed and to flow naturally on screens of different sizes, with embedded clickable 'links' that would open up documents on any server accessible on the Internet. Documents were 'marked up' with 'HTML' tags, that were inserted directly into the 'source' text of a document. HTML tags indicated characteristics such as bolding, italics, layout within rows and columns of a table, etc. 'Opening' and 'closing' HTML tags simply surrounded any enclosed text, to adjust the way that text should appear on screen, or for example, to indicate a 'URL' (Uniform Resource Locator) address at which another 'linked' document could be found. Clicking on a bit of linked text (text surrounded by '' and ' tags) in a document, would load that linked document from the server machine located at the specified URL address.
Documents on the Internet can be found at a specific 'IP' address, which is a 4 segment number referring to the physical address of a computer attached to the giant world wide connected network of machines that make up the Internet. To make addresses of documents easier for users to remember and find, the URL system was devised to be used in browsers. When you type a URL into your browser's address bar, the browser program sends a request to a 'DNS' server, which is typically operated by your 'ISP' (Internet Service Provider, such as Comcast or Verizon). When a web site owner registers a 'domain name' (somesite.com), the 'registrar' company (such as Godaddy) updates the databases of DNS servers at ISPs all around the world, to 'resolve' that domain name to the current IP address of the computer where the domain owner stores documents which are to be made available at that domain. Try typing '142.251.16.138' into your browser's address bar. You'll see that's the IP address which the URL 'google.com' (www.google.com) resolves to (at least in the area of the world where this author is located).
Once a DNS server sends back the IP address of the URL you've typed into the address bar, your browser then sends a request, using a data protocol called 'http://' to the computer at that IP address. To be a bit more specific, each IP address actually has 65,535 connection 'ports' available to send and receive data, so many 'client' programs can be sending/receiving info at the same time, between different 'server' programs running on a single multitasking computer at a single IP address. HTTP requests are traditionally sent over port '80', so the actual network address of Google is really http://142.251.16.138:80 (notice the beginning 'http://' and the ending ':80'). The browser simply knows to attach pieces such as 'http://' and 'www.' to URLs and ':80' to IP addresses when connecting to computers that are serving web pages.
On the computer at the desired IP address (the URL of the web site being loaded), there is a program called a 'web server' running. This program is most often 'Apache' on modern Internet web server computers (but there are many other web server programs running on server computers all across the Internet). The purpose of web server software is to accept requests from client software (your web browser), and send back the requested data. When your browser connects with the Apache program on a server computer at the web site's IP address, Apache's default response is to send back a file called 'index.html'. Your browser receives and loads the contents of that index.html document, and displays the document in your browser, as specified by the HTML tags in the document. That index.html file typically contains text with links to other HTML files on the server hard drive. If you type in the name of some other document (www.yoursite.com/filename.html), Apache will send back that other 'filename.html' document to your browser. Web developers can create as many different named HTML documents as they want, and store them in the 'public_html' folder on their server machine's hard drive (public_html is the home folder designated for Apache to serve from), and then link those documents together with HTML link tags in each of the documents, to create an entire surfable web 'site'. Subfolders can also be created on the hard drive of the server computer, so that if a browser requests 'yoursite.com/subfolder/', Apache will serve the client's browser the index.html file (or any other specified file name) from the '/subfolder' directory on the server machine's hard drive.
Any of the documents on a web site can link to documents on any other publicly available server, at any other domain URL. When the user clicks on any link in an HTML document displayed in their browser, if the link points to a server at a different URL domain, the browser simply requests the indicated file from the URL at that link, and performs the same process of interacting with the Apache server, and requesting documents from the machine at that IP address. In this way, users of a browser can click between hyperlinked documents anywhere on the world wide web, without needing any technical knowledge of computing or networking, to jump between documents on any machine connected to the Internet. The whole system can be thought of as a point and click 'GUI' (graphical user interface) for documents delivered by web server software on the Internet of connected machines all over the world.
It's important to understand that anyone with a computer, an Internet connection, and some free web server software, can create a website. No other special hardware/technology is required. Apache runs on any Windows, Mac, or Linux desktop, and there are many pieces of web server software which run on mobile devices, and even within embedded machines (i.e., in automobiles, TVs, specialized hardware such as guitar amplifiers, etc. - little servers built into devices, form the basic idea behind how something like a modern smart fridge connects to an Alexa, and can be controlled over WIFI with a cell phone). To start your own web site, all you need to do is download Apache onto your home computer, create and save some .html files to the public_html directory on your hard drive, and tell people to go the IP address of your computer on the Internet. You can buy a domain name for a few dollars per year from Godaddy, point it to your computer, and your web site will work exactly the same way as any other (of course, if it gets busy, you'll probably need to switch to 'hosting' your files on a rented server at a dedicated data center which can handle the 'bandwidth' (network traffic) use).
Note that if your computer is behind a 'firewall', (for example, attached to a WIFI 'local area network' behind a 'router' that connects to your ISP), then you'll need to 'open' port 80 in your router so that client connections comining into your router's IP address on the Internet (www.your.router.ip.address:80) get 'forwarded' to the IP address of your computer running Apache (or whatever other server software you use), on your WIFI network (often 192.168.1.) - you can look up exactly how to do this in your router's manual. You can also do this for any other network server application which you want to make available to the public. For example, 'email' server programs typically work on ports 25 or 26 for sending email data, and ports 995 or 110 for receiving email data. Email server and client programs send and receive the contents of email documents using established data protocols (formats for packaging the data), over the same established networking protocol used to transfer web page information (TCP). If you wanted to develop your own email server/client programs with some programming language, for example, you could look up the specification for SMTP (simple mail transfer protocol), POP (post office protocol), IMAP (Internet Message Access Protocol) and follow the guidelines which specify how to send that formatted data over the TCP (transmission control) protocol, which defines how data packets should be sent over a network connection (the same network protocol used to deliver data on the Internet in HTTP:// format). The fact is, most modern programming languages have the ability to use established network protocols built in, so writing code to send emails is as easy as using some sort of 'send_email' function. Note also, that you could choose to run server software to operate only within your local area WIFI network, to share private files among your personal computers, for example. You can even run server software on a single machine, and access it in a browser on the same computer at 'http://localhost'. So, although you don't ever need to create any complex server or client software to run a webpage (just run Apache or some other free, pre-made server software, and write some HTML documents), the basics of how every part of the entire system work, can be learned and put to use with just some basic learning. If you want to take part in building pieces of a new specialized server software (or really any other type of software), there are thousands of open source projects that you can take part in online (of course you'll need to learn a programming language to do that).
During the 1990's, a proliferation of 'search' engines such as Yahoo, Altavista, Lycos, Ask Jeeves and others, were predecessors of Google, Bing, and Baidu. Search engines are programs that run at the largest data centers on the planet, to 'crawl' links on known web sites, in an attempt to eventually 'index' (download) every single document on every web server connected to the Internet (i.e., search engines actually attempt to store all the web pages on the Internet, on their own hard drives). Using algorithms which assign importance to each document, based on the number of other prominent web sites which link to a given page, and the relevance of the document's content to 'keywords', the search engine's home page allows users to type in search terms, and then provides a list of links to web pages, with results ordered by relevancy. This continues to be one of the greatest income producing services on the Internet, as businesses of every type pay money to market their products and services online via search results. 'SEO' (search engine optimization) is the process of setting up a web site's content, and performing marketing related activities, to raise the site's position in search engine results.
'E-commerce' web sites such as Amazon, Ebay, and Etsy allow users to search specifically for items to purchase, and most large retailers such as Walmart, Best Buy, and Home Depot now provide online 'shopping carts', which allow customers to view details about items, collect items to purchase, and pay via credit card, Paypal, Venmo, Zelle, and other online financial services.
HTML has evolved dramatically from its initial incarnation, so that it now can display text in any font/size/color, along with images, sounds, video, etc., and everything can be displayed with more detailed and specific layout options. The style (colors, sizes, fonts, etc.) of an entire web site can be specified in 'CSS' (cascading style sheet) files, which are imported at the beginning of an HTML document. This enables changes to be made to a single style sheet, and that then affects the look of every document which imports the CSS file. 'JavaScript' language code can also be included in an HTML document, to allow complete programs, with logic, calculations, loops, etc., to run right in a web page document. JavaScript that runs in a browser is called 'client side' or 'frond end' code. Originally designed to perform simple actions such as animating button presses, JavaScript has evolved to the point that it can now be used to create full data management apps and games which respond to keyboard/mouse/game controller input by the user, and it can even display 3D and virtual reality graphics, all running directly in documents displayed by a browser. In order to make such complex things happen in a browser window, there are entire front end 'frameworks', built upon JavaScript and the other native components in a web page, which enable easier, more productive and more performant interactivity in web based apps. 'React', 'Angular', 'Vue', 'Svelte', 'Ember' and other frameworks allow user interfaces to be designed in ways that are less awkward and more effective than with plain HTML, CSS, and JavaScript alone. Most of the well known languages these days can actually be compiled to JavaScript, or interpreted by frameworks that are written in JavaScript. For example, 'Skulpt' allows Python code to run directly in a browser document, and tools such as 'Streamlit' allow developers to build complete interactive web pages entirely in Python, so that developers who prefer that particular language never need to touch HTML, CSS or JavaScript at all. 'WASM' (Web Assembly language) allows programs written in C to be compiled and run directly in a web page, with performance that is close to running C directly on the metal of the computer hardware. This has enabled many complex programs which had been developed to run in traditional operating systems, to run directly in web pages. Extremely complex video games have been ported to run directly in the browser, and deeply capable office software such as Google Docs (which has replaced Microsoft Office in many environments) have been written from the ground up to run in modern web browsers. Frameworks such as 'A-frame' and 'LOVR' allow 3D and virtual reality apps to be built and run directly in the browser. What this means for users is that it doesn't matter which operating system or hardware a browser is running on (desktops, mobile devices, etc.). Users of web apps simply need access to a modern browser, on any computer. Much modern software is built upon the browser, in just the same way that previous generations of software were built upon operating system APIs. 'Chromebooks' are in fact complete computers built primarily to provide an inexpensive machine in which most of the software runs in a browser. The web has indeed become one of the most pervasive 'platforms' for modern computing.
MOBILE DEVICES
The standardization of software development tools, protocols and data formats, as well as the miniaturization of hardware has led to a proliferation of 'mobile' operating systems and devices in the 2000s. iPhone and iPad devices made by Apple run their proprietary 'iOS' operating system. 'Android' devices manufactured by Samsung, Amazon, LTE, Huawei, ZTE, Alcatel and others, use the open source Android OS created by Google. These devices all tend to include a unified package of miniaturized standard hardware such as touch screens for GUI I/O ('input and output'), microphones and speakers for audio I/O, WIFI network and cellular chips for wireless Internet connectivity and voice calls, cameras for image and video capture, 'Bluetooth' for wireless peripheral hardware connectivity, microSD cards for removable data storage, as well as fast CPUs, RAM, and other basic componentry. Mobile operating systems were designed from the ground up to natively support all the most common modern data formats (images, audio, video, documents etc.), as well as all the essential modern protocols for network connectivity, and other features which have become commonplace/essential in modern computing. They are built in a way that eliminates much of the legacy 'cruft' that burdened earlier hardware/software systems, and optimized to boot quickly and use the least amount of power possible when performing any software activity. At the beginning of the mobile revolution, this meant that mobile devices often only enabled minimally capable software apps, stripped down web browsers, and other imposed usage limitations. For example, web sites originally designed to be viewed on desktop machines were awkward to use on mobile browsers, but this situation has changed dramatically, and mobile devices have moved to the center of modern computer use.
'App stores' allow mobile programs to be downloaded and installed easily, with security and permissions built into the core of the system, so that it's far more difficult for nefarious code and viruses to be installed unintentionally by average users (that sort of security problem presented a much larger risk on earlier desktop operating systems). Full office suites, virtual reality systems and 3D games, complex artificial intelligence systems, and every other sort of rich software is available for mobile devices, and because there are so many billions of devices being used around the world, the cost of software and hardware designed for mobile systems has been reduced drastically by economies of scale. Modern mobile phones and tablets which cost less than $50 have far more computing power, and contain more complex/useful componentry than was available on expensive desktop computers a decade ago. And as a result of lessons learned from decades of user interface (UI) design, they tend to be easier and more natural for humans to use.
The proliferation of mobile device 'platforms', as well as inexpensive WIFI/Cellular data service, and the popularity of social networking and online shopping - that potent combination of developments has changed the dynamic of how humans interact with each other, and the nature of how we accomplish our most basic activities. Being constantly connected to any information, able to communicate with loved ones, and able to connect with hardware in our homes, cars, and elsewhere has forever altered how we work, entertain ourselves, and satisfy our most basic human needs and desires. Constant connectivity has also shifted the paradigm of software and hardware itself to rely more upon connections with server systems that run 'in the cloud'.
BACK END (SERVER) TECHNOLOGY
Front end code that runs in the browser, is just one part of the machinery of modern web apps. Apache (and other web server apps) are no longer the main stars of the show on web server computers. These days, most browser documents are created 'dynamically' when a user requests information, instead of just being served as 'static' (unchanging) documents. When you request a .php file (http://yoursite.com/myfile.php), for example, what Apache actually does is run that code file through the PHP language interpreter on the server, and the code in that PHP document is evaluated and in turn used to output an HTML document which contains current data. You can include a PHP tag in your web page, for example, to evaluate and insert the current time, into the document that gets delivered to the client (there's no way to do that with just traditional static HTML in a document file). But of course, you can perform far more complex server calculations with server languages such as PHP. Server languages can accept data typed into GUI forms submitted by users, via a browser, from a web page, and enter that data into a database, or perform conditional evaluations about what the user should be shown next, based upon data they've entered. Server code can, for example, perform extremely complex computations upon data selected from a live database, and create graph/chart images from those results, and then insert those images into the document returned to the user by Apache. In that case, Apache is just doing the serving of the final document - the dynamic creation of the data in that document is handled by the programming language on the server. Data can be sent over a network connected to a server, or peripheral hardware attached to the server machine can be controlled (imagine, for example, a 3D printer which can be controlled by requests sent from a browser) - basically any operation which can be performed via any programming language on any computer, can be performed by a server computer connected to remote users, who send requests to the server via their browser. And PHP is just one of hundreds of popular languages used to process data on servers, and send results back to clients. Python, Java, JavaScript, C#, Ruby, and just about every other language you may have heard of, all have standard methods of being incorporated within backend web server technology ('CGI' was the first such method), and often features of languages and their ecosystems are mixed within a single server application. Many languages have specific strengths. Python, for example has been used extensively in the artificial intelligence and data science fields, so its ecosystem is filled with many mature code libraries used to perform statistical analysis, image recognition, natural language recognition, etc. As a result, the creators of web sites built around processing that sort of data may very likely want to use Python on the back end to process requests from, and send results back, to clients.
Whatever language is used to process data, most information is stored on servers in a 'database' (searchable, sortable, and otherwise manipulatable tables of info). Each database management system (DBMS) has it's strengths, but the most common ones, MySQL, PostgreSQL, MS SQL, Oracle, and others traditionally use 'SQL' as the 'query' language, and most often run on server machines using the Linux OS. SQL looks a lot like readable English text, saving and retrieving info in 'tables' (rows and columns) of data, with commands such as "INSERT INTO Customers (Name, Address, Phone) VALUES ('Joe Smith','1 Street Rd','555-1234');" and "SELECT * FROM Customers WHERE Name='Joe';". In recent years 'NoSQL' (non-tabular, non-relational) database systems such as MongoDB, CouchDB, Cassandra, Redis, and others have become popular.
'Full stack' software development tools such as Ruby on Rails (developed in the Ruby language), Vaadin (Java language), Anvil (Python), and NodeJS (together with JavaScript in the browser) form complete front-to-back-end systems, often with native language 'ORM' systems that handle database calls without requiring any SQL querying. For example, software developers can write all the code which runs in a browser, and on the server, and which connects to the database, entirely in Python, using a toolset called Anvil. These sorts of full-stack systems are popular because they enable developers who have previously only focused on front-end, back-end, database, or other specialized pieces of the client-server Internet software development process, to now apply their expertise in any one part of that process, to the entire development cycle.
Modern server interactions don't end with client applications that run in web browsers. Requests to web servers can also be sent from any client app running on any operating system which can connect to the Internet. An app running on your iPhone or Android phone/tablet, a desktop app, even embedded apps within connected devices such as Alexa, your automobile, or even the lights and appliances in modern houses, etc. - can all send/receive data to/from servers on the Internet. Many mobile and desktop apps are actually just front ends of web sites packaged up and distributed as standalone 'webview' apps (webview apps package a browser, along with the front end client code into a single package).
Many web services now offer 'Rest APIs', which provide functions that can be run by sending 'GET', 'POST', 'PUT', 'PATCH', and 'DELETE' requests over http:// protocol. A client program of any kind, for example, can request weather data from a weather web site, by submitting an API request to the server, including for example, a specified date/time range and location information, all sent to the online API function as a long string of text. The online API functions called by this request (on the server) then return just the requested data to the client. APIs such as this are typically meant to send back only essential info, as opposed to entire formatted web pages, so that the pure data can be used in apps of any sort. Some services charge fees for valuable data which their APIs provide (especially financial and business data). Some companies simply provide access to user account services via an API, so that third party developers can provide special features and views/interactions with account data, which have not been considered by the developers of the web site. For example, Etsy users may want to add multiple items, or make bulk changes to prices/descriptions of their inventory in the Etsy system (think perhaps, special sales events), in ways which the Etsy.com website doesn't currently enable. To help with these sorts of activities, apps for Android and iOS have been created by third party developers, which connect to the Etsy API, log into a user account and make such changes, in ways which haven't been implemented by Etsy. This is a win-win for everyone involved: busy Etsy sellers get to use an app that lets them more specifically control their inventory, or perhaps generate special sales reports, etc., Etsy doesn't have to spend its resources replying to special development/usage requests from small groups of users, and third party developers make money selling an app that caters to that crowd.
Modern web apps that run in a browser now actually follow the API model above, more often than not. Instead of constantly submitting page requests to a server, and having the server reply with a completely new, dynamically created full web page (with all its heavy graphics and layout code), it's more common to serve a 'single page app' (SPA), and let that single page perform much of the work of interacting with the user, on the client machine. SPAs are rich interactive web apps, in which the user typically interacts with JavaScript (JS) code (or some other language that compiles to JS) running on the page. When the user clicks a link, or submits some text using a form on the page, the front end JS code sends an API request to the server (requests like this were originally called 'AJAX'), and makes some changes to the display of data on the web page, based on the data returned by the server, without ever reloading the majority of the page. This saves a lot of bandwidth and server load, compared to re-downloading an entirely new page every time some data is requested from the server. SPAs are also a bit more performant for the user - more like 'native' apps. Instead of watching/waiting as a web page is redrawn on screen every time a request is made of the server, just the data is refreshed. And because the same sort of API model can be used to interact with data transactions between server and client code, the development of 'cross-platform' apps is made much simpler. A front end can be built for web, mobile, and desktop interfaces, and each of those front end apps simply interacts with the exact same back end server code exposed by the Rest API. This whole paradigm of connecting to a server, where all the most important data storage and processing occurs, has shifted the importance of client platforms - as long as a platform can be used to connect to a necessary server, it doesn't matter as much which platform a user chooses to interact with on the front end.
The connected 'IoT' (Internet of Things (connected smart devices)) makes use of the most cutting edge advantages in software and hardware technology, at every level of client-server interaction. Modern devices of all sorts can send data from cameras, sensors, and every type of information-gathering interface, to be collected and processed by machine learning algorithms, natural language processing, and other types of artificial intelligence code on servers.
'Maker' systems based on 'Raspberry Pi', 'Arduino', and other inexpensive standalone microcontroller boards (typically attached to a PC by USB cable), are programmable in well known languages such as Python and C++, and supported by enormous ecosystems of manufacturers who provide every sort of imaginable specialized componentry (not just electronic pieces, but also 3D printed hardware which can be integrated with electronics and mechanics for specialized uses). Communities of users provide tutorials and examples of how to build hardware devices, robots, etc., from those basic components and language tools, demonstrating how to connect those devices to the enormous ecosystem of server resources online, accessible through pervasive wireless and mobile Internet connections... all that enables normal people around the world to easily bring to life new custom inventions which could not have been imagined a few decades ago.
CLOUD SERVICES
As the load on servers increases, with the entire world connecting, new technologies are keeping up with the way servers handle traffic. 'Cloud' computing services manage server hosting using 'container' technologies such as 'Docker' and 'Kubernetes'. Google Cloud, Amazon AWS, Microsoft Azure, and many other commercial services provide 'virtual' machine hosting. Developers write their server software as if it's going to be run on a single physical computer, then create a configuration package which represents the entire machine - it's physical design, operating system, language backend with all the necessary code libraries installed, etc. - and then install that configuration on a cloud service, where it runs as a virtual machine which can be duplicated ('spun up') whenever traffic increases. This provides a hands-off way for developers to run their software, in the same way they would on a traditional desktop development machine, just within in a virtual environment which can keep up with enormous loads, by leveraging the huge server farms and physical infrastructures owned and operated by large companies like Google, Amazon, and Microsoft. Cloud hosting services charge according to the amount of computing power and bandwidth a server uses, so when there's little traffic, there are very few costs associated with a virtual server. There is also very little initial investment required to install and start up virtual servers in the cloud (no new room full of physical computers, high speed ISP service, or expensive crew of IT professionals needed to manage it all at a privately owned physical data center). So, start up costs, usage fees, and management troubles are low for virtual servers that aren't busy, but because cloud services are operated by huge companies with enormous physical and technical resources, already built to run at the scale of the Internet, it's far easier for them to scale up virtual services to any particular web server renting resources on their system, as that particular server's needs increase. Cloud services can reliably handle spikes in traffic at times such as Christmas, when shopping web sites get slammed with tremendous buying activity. They simply spin up more copies of the virtual server, as needed (a task which would be incredibly expensive and difficult to do at a private data center with real physical machines).
Many servers still do run on physical machines in private data centers, on 'shared web host' services, and even on personal computers in someone's house or business. It's just as easy as ever to load Apache web server software on any Windows, Mac, or Linux machine, to buy a domain name for a few dollars a year and start a web site to share some info about your business, or to write a server API to remotely control some hardware you own, etc.
MODERN SOFTWARE DEVELOPMENT ECOSYSTEMS
"It's all Code". <- That idea is what makes computers so malleable and endlessly useful. The fact that their functionality can be *changed, that humans can create, manipulate and perform calculations upon any sort of useful data (not just text and numbers, but images, sound, audio, video, virtual reality environments, etc.), and control hardware and the very electrons that run them (literally at the atomic level), *with human language* (code) - that's one of the things that makes the capabilities of computers so magical! The ability to write code to control every piece of hardware and data that a computer touches, is at the heart of everything interesting and useful in computing.
Java, Python, JavaScript, C/C++, a variety of languages and tools created by Microsoft (C#, Visual Basic), and the entire stack of front and back end web development languages/frameworks (JS in browser, sent over HTTP:// protocol, to Apache or other web server app running at a URL attached to a network IP address, with data processed by PHP or some other language running on the server, and saved to files or some database), as well as SQL (which is used to interact with most database systems), form the overwhelming majority of languages and tools used in modern software development.
These days, most programming languages share a majority of common features, and experienced developers will choose a particular language which is 'right for the job'. Once you learn the process of writing code in one language, switching between languages is not so much of a challenge. Most languages use:
- 'variable' words to assign labels to changeable data values
(x = 10)
- 'functions' (or 'methods') to perform actions using variable data
(print x)
- some variety of list data structure (collections of values) to contain and manipulate multiple pieces of data ('arrays', 'lists', 'series', and 'dictionaries' (lists of associative key:value pairs) are some common types)
(x = {'joe', 'bob', 'tom'})
- conditional evaluations to respond to user input or some other situation/state of data
(if somevariable = thisvalue then: )
- 'loops' to perform repeated processing upon each piece of data in a list
(for i = 1 to (length of list): )
- data persistence
(write data.txt x) (or saving to a database or files on the hard drive)
- some method of providing user interface (UI) interaction with a human - most often via a graphic user interface (GUI), or sometimes just at the command line in simple utility apps.
There are a variety of basic approaches to organizing code into 'procedural', 'object oriented', and 'functional' patterns, and there are 'dynamic' and 'static' ways of naming variables (which dictate whether the type of data a variable holds needs to be defined before the variable can be used), which can change the way code looks in different languages, but the basic thought processes in each language are generally very similar - it's mostly about using variables, functions, conditional evaluations, lists, loops, databases, and providing some form of UI interaction.
Most coding these days happens in 'IDE's (Integrated Development Environments), which have evolved from simple text editors, to intelligent coding environments that suggest and autocomplete code snippets for the developer, check for coding errors and alert the human developer of potential fixes, provide easy management of code files in large projects, and allow the developer to easily jump back and forth between running, debugging, and editing pieces of code, until it works as expected.
The code for modern open-source projects is typically saved in a 'Git' repository. Git software allows teams of developers anywhere in the world to share the work of writing and making changes to code, by managing the potentially confusing process of having many people all making potential changes to the same code files at the same time. Git allows managers of a project to approve and merge changes to files which have been submitted by many people, roll back code to any previous version of a file which existed in the past, and provide a common location and interface (sites such as github.com) for users to easily download and 'fork' their own versions of a project, resubmit useful changes back to the original project's code base, etc.
The reason that any current language is chosen for a given job, now tends to have less to do with its specific language features, and more to do with the 'ecosystem' that surrounds it. Some languages provide superior support for scientific, engineering, machine learning, and artificial intelligence endeavors (Python, R, Wolfram), while others have traditionally been used to write desktop apps with rich interfaces that run across OS platforms (Java, variations of Visual Basic). If you want to write front end code for a web page, you need to use JavaScript, or some language that can be interpreted by, or compiles to, JavaScript or WASM (because those are the only fundamental language tools that run in browser windows). Some languages such as LUA are meant to be used as small, fast, embeddable scripting tools, easily incorporated into game development engines, for example. Some languages such as VBscript are meant for specialized purposes (scripting spreadsheets), for example. C and C++ are still at the heart of virtually all 'systems' development - software that works works close to the metal: hardware drivers, OSes, programming language interpreters and compilers, embedded software in cars and machines, large apps like browsers, and any other software which must perform with every bit of speed a machine is capable of, and upon which other apps can be built - that continues to be the domain of the C family of languages.
The 'libraries' of functions and 'frameworks' of functionality available to a given language, the platform(s) on which they run, as well as the culture/history within a given company or community in which it's used, will often be the deciding factors as to which language is used for a given project.
Python has become incredibly popular in recent years because it's easy for beginners to learn, but powerful enough to be used for almost any sort of common development work. Once a programming language like Python becomes popular enough to be used by millions of developers, then there is lots of 'support' for it among a community of developers who extend its features and reach. 'Makers' can use Python to write programs for Raspberry Pi boards, to develop robots and other custom mechanical devices. Web developers can use it to build front and back end systems all in one language (Python is installed on most web server machines by default), data scientists can use the multitude of mature Python code libraries for analyzing data and applying artificial intelligence to real world problems (with Tensorflow, PyTorch, Theano, OpenCV, etc.). Python makes it much simpler to perform that work, with much less time and effort required, and much more potential capability, than with other languages that don't have support for those libraries of advanced functions.
Many languages have 'package managers' which allow developers to easily import such libraries of useful functions for their language of choice (Python has 'PIP/PyPI', Java has 'Maven', JS has 'Bower', Node.JS has 'NPM', PHP has 'Composer', etc.). Code libraries are often the result of deep research and development, and the functionality/capability they add can dramatically improve the possibilities of what any developer can achieve with a language. By simply 'importing' a library into a line of your code, new super powers can be added. Libraries generally provide functionality for operations such as analyzing business data for buying trends, producing charts and graphs, enabling image recognition and natural human language processing, adding user interface widgets and interactions, video conference capabilities, etc. 'PyGame', for example, is an entire collection of functions which make it easier to build games in Python. Just type 'pip install pygame' on a computer that has Python installed, and type 'import pygame' in the top line of your source code file, and you can write code to create games that run on all sorts of operating systems (Pygame enables easy placement, movement and collision detection of graphics on screen, playing of sounds in response to on-screen events, etc.).
Libraries are created by the community of developers who use a language, whenever they devise solutions to common problem domains encountered while using the language. Node.js, for example, is an implementation of JavaScript that typically runs on web servers, so the libraries available to it are generally geared towards helping developers add more functionality to the back end of web apps. Other JavaScript libraries which are meant to run in browser documents, on the other hand, tend to help improve layout design and user activity on the front end.
Java has a massive built-in API (library of functions) which are well suited to building cross platform enterprise apps, with strong support for security and parallel processing features (to make best use of multiple core processors), and it has the strong financial backing of a leading company in the tech industry (as well as the long term staying power of being open source, in case that company ever goes out of business), so it attracts developers who need those enterprise features.
Microsoft's suite of languages, including Visual Basic and C# have long been a popular choice for developing applications for MS Windows operating systems, because they were designed from the ground up to work tightly with every feature of that OS (obviously, because both Windows and those languages are both made by the same company). In the past few years, however, Microsoft has taken important steps to support open source and cross platform development.
PHP has long been used for developing web site back ends, because it was built from the ground up to support a massive library of functions for making simple work of typical web site related tasks (logging in users, saving data submitted from HTML forms to online databases, etc.).
Some languages have become popular just because they were used to create a particular 'killer app' (wildly popular application). Ruby is used primarily for the 'Ruby on Rails' framework, which enables full-featured full-stack web sites to be developed (front end, back end, database code, and all the pieces in between) very easily, with very little time consuming configuration, and lots of typical features used on web sites, built-in (so that no other tools are required to build the full stack of front and back end components of many common types of web sites).
Virtually every programming environment provides basic capabilities for performing 'CRUD' (Create Read Update Delete) operations upon data 'structures' (lists of data items). The term CRUD is used to refer to the management of useful information - with ways for users to input, store, search, sort, calculate/process, transfer and output that info. CRUD operations form the basis of most useful business and personal information management tasks which have made computers in all ecosystems so useful - think scheduling, inventory, billing, point of sale, communications, and other productivity-related human activities. Even social networking web sites are fundamentally built around CRUD capabilities (creating, saving, and sharing pieces of personally interesting data). These days, most CRUD interactions are handled by a database system and SQL code on a server, and programming languages are employed more for logical decision making and calculations upon data, but those lines can be drawn in many different ways, depending upon which libraries, frameworks, and tools are used. For example, database systems may be hidden by an 'ORM' language layer, which allows for CRUD transactions to be perform entirely within a natural dialect/extension of a programming language (using variables, functions, objects, and other familiar code constructs).
When it comes to developing games, virtual reality, and other media heavy apps, it tends to be the tool set, more than the language that really matters. 'Unity' is by far the most popular game 'engine' because it allows developers to easily design 2D and 3D scenes, add animation to graphic models, and control interactions which are common in games (scrolling across screens of tile maps, jumping player characters between visual platforms, detecting collisions between moving graphics on screen, etc.). Unity also makes it simple for game developers to publish their games across mobile, desktop, web, and game console platforms (XBox, Playstation, Switch, etc.). 'Unreal' and 'Cryengine' are used in many high quality 3D games, and even mainstream movie CGI. 'Godot' is becoming very popular because it is fully open source (totally free to own and make changes to), it creates high quality 2D and 3D games, and requires absolutely no payment of royalties when a game made with it becomes commercially successful (Unity, Unreal, and Cryengine require developers to sign a contract, so they receive significant royalties from game sales made with their engines). Although C++ has been at the heart of game development (due to its fast performance), each of the big game engines support multiple programming languages, so developers can write code using the structure and paradigm they're most comfortable with (static/dynamic variables, procedural/functional/object-oriented patterns, familiar syntax such as the use of white space (indented lines), semicolons, etc.). There are many other small game engines which have been successfully used to create commercial games, in virtually every popular programming language (JavaScript Phaser, Lua Corona SDK, Java libJDK, Python PyGame, etc.).
One popular trend in recent years has been the growth of 'low-code' and 'no-code' software development tools. Companies such as Bubble, Appian, Mendix, Outsystems, and many others provide simple to use IDEs which enable less technical users to create front ends (mostly drag-n-drop GUI forms (text fields, drop-down selectors, images, etc.)) that connect to database systems and perform common business logic operations, all with little to no code required. These sorts of app development systems (mostly CRUD related, as opposed to game or media related) can provide businesses a way to create useful apps which are more powerful and customizable than spreadsheets, for example, and capable of providing useful functionality to automate business data management tasks (accounting, inventory maintenance, sales, communications, etc.), without needing to hire expensive software development firms who may not have as much experience as workers in the profession they write software for (after all, most software developers spend time writing code, rather than experiencing other domains of work - so they may not truly understand the demands placed upon users operating the software they create). Low-code and no-code tools allow professionals within any field of work, to create truly useful custom software that exactly fits their operational needs, and helps to cut down on business costs, ease work difficulties, improve customer retention, and add money to the bottom line in effective ways.
'Scratch' is a software development tool which has become very popular in education. It allows users to visually click together 'blocks' of code, which represent traditional code structures such as variables, functions, conditional evaluations, and loops - but without having to type any code, remember the syntax of any code structures or function names, worry about misspelled words in code, misplaced semicolons, etc. This system has proved to be very useful for teaching beginners and even young children how to build software applications, and other systems such as MIT App Inventor and Stencyl use the same basic block syntax to enable the creation of more powerful cross platform apps.
No-code tools are also available in the most popular game engines. Unity, Unreal, and Godot include 'visual' programming languages, which enable the use of graphic diagrams, flow charts, and visual data connections, to control program flow and interaction, so that less technically interested people can create visually stunning, immersive, interactive 2D/3D games and apps for virtually any platform, without ever typing a single line of code.
For serious developers creating mobile apps, using the standard software development/publishing process generally requires a few additional specialized steps. Because the app store model is the primary method of distribution for mobile software, apps must be submitted to, and comply with, each app store's requirements.
The default file type for Android application executables is .apk ('Android Package'). Android software is based on the Java language, with additional native support for Kotlin, which is a higher level language that runs on the Java 'runtime'. Because the Java runtime is typically implemented in C and C++, there is also lower level support for C and C++ code (you can work directly with machine code on Android using these languages). The Android 'SDK' (software development kit) is the primary Java interface that provides the full set of Android OS APIs (functions that software is built from). The 'NDK' (native development kit), provides access to C and C++ APIs. The NDK enables 'porting' of code written for other OS platforms. Because most other programming languages tend to be implemented in C/C++, the NDK offers a way for code written in other languages to be run on Android (i.e., programming language compilers and interpreters can be 'ported' to run code on Android). The JNI (Java Native Interface) allows code written in other languages to communicate with and access all the functionality of the Java functions that make up the complete Android API.
Code for Android is typically written on a desktop PC where the Android SDK/NDK have been installed (most often using the 'Android Studio' IDE, which runs on Windows, Mac, and Linux), and then compiled to an .apk executable file, which can be transferred to an Android device or uploaded to the Google Play Store. There is an evaluation process required to upload your app to the Play Store, and Google developer accounts must follow Play Store rules, in order for apps to remain in the Play Store. One of the primary goals of app store evaluation is to ensure that apps are secure (don't contain viruses or privacy breaches, and can't otherwise take control of the Android device or cause harm to Android users). When an Android .apk is installed (either through the Play Store, or downloaded from another website, installed from a removable memory card, etc.), the Android OS asks the user permission for the app to use services on the device such as access to the file system, contacts, network services, etc., which could potentially give the app control of your data/machine. If you don't want the app to have any potentially risky security access, you can stop the installation, or later uninstall the app completely.
The default file type for iOS is .ipa ('iOS App Store Package'). iOS software is originally based on the Objective-C language, but Apple now recommends their newer Swift language as the native default. iOS also supports C/C++, to enable porting of other software and programming languages from other platforms. Code for iOS is typically written on a Mac, where the Apple Xcode environment has been installed (Xcode doesn't run on other platforms - Apple Macintosh only). You can create code projects for Xcode using other software development languages/environments, but generally that code eventually needs to be compiled by Xcode, to run on iOS devices. Submission requirements for Apple's App Store are even more stringent and time consuming than for the Google Play Store.
Apps which you buy in the Play Store can be installed on any device tied to the Google account which purchased the app, and the same is true for Apple's app store and iOS devices. This makes it easy to move all your apps to a new phone/tablet - just login with a new device and download any apps owned in your account. Google and Apple both provide many other services such as contacts, calendar, image/video/document creation and backup, web site password management in their browsers (Chrome for Google and Safari for Apple), map and location history, etc., which make it easy to access all your personal data on any Google or Apple device, for as long as you maintain an account with their cloud services.
SOME MORE NOTES ABOUT SECURITY IN MODERN COMPUTING SYSTEMS
(TODO: running secure processes on servers, rather than clients, Captchas, OAuth, session management, beating DOS attacks, injection attacks, viruses and other malicious code)
UNDERSTANDNG HOW NEW LAYERS ARE BUILT UPON OLD LAYERS, AND WHAT *TO DO*, TO BUILD TECH SKILL
It's important to recognize how many of the 'new' platforms and services that abound today are made of not-so-new technology. They all rely on operating systems (mostly Linux based), which have had much of the 'legacy crud' (no longer useful code and features) removed, updated to look nice and to run efficiently, and installed in small devices with all the most standard components (CPU, RAM, hard drive, microcontrollers for wireless network, camera, audio hardware), based on all the same traditional hardware tech used in older machines, just miniaturized and controlled by a GUI interface that uses a touch screen rather than a mouse and keyboard for user interaction. You can still type commands at the console prompt in both Android and iOS, just like users did on DOS machines in the 1980s - or even run game cartridge ROMs in 'emulator' software, or BASIC code in an interpreter app to control the system hardware, just like on the first hobbyist machines in the 1970s. Specialized standalone hardware of almost any type (machines used in manufacturing and other industries) typically contain computers, or can connect to computers through USB, serial, or parallel cables, or to a card installed in a bus slot in a PC - and then can be controlled by the standard hardware and software components in a computer.
If you become involved in doing technical work, the exact same knowledge that was required to fix problems or to create new functionality for old machines is primarily still the same for new machines. The pervasiveness of new devices has certainly exploded, and because of this growth, new ecosystems have been created - and old OSes like Windows have been updated with a focus on connectivity and security. Growth has also brought the value of pervasive data front and center in our natural lives - we communicate with our loved ones, do much of our work, and find most of our entertainment online. But most of what modern users consider 'new technology' is not really even technology, but rather implementations of existing technology which have found new ways into our daily activities: Facebook, Twitter, LinkedIn, Instagram, Snapchat, TikTok, Pinterest, Reddit, YouTube, Amazon, Hulu, Zoom, WhatsApp, and the social communities/business activities which surround those companies' services. The foundations of computing technology that compose all those online services, still make use of all the same hardware and software pieces that have been around for years.
What you actually *do* today, to make use of computing technology, is to interact with some cross-section of the pieces of hardware and software technology that have been described in this document. For example, if you're an 'average' user, you may just buy an Android or iOS device and download/run apps from the Google and Apple stores, to connect with your favorite social media services, to play some games, to do some work with business apps, etc. Or you may buy a Switch gaming system and purchase game cartridges to plug in and run. Or you may use a desktop computer to browse web sites.
If you eventually want to build your own web site, though (hehe - take a deep breath and get ready for a long ride), you may likely end up using a program to visually layout web page documents, and to upload the generated HTML pages to a shared hosting service provider, to be served at a domain name which you buy from GoDaddy. Perhaps along the way, you may use some image editing software to crop and adjust photos to be inserted into the HTML pages - perhaps you'll learn to convert those images between JPG and PNG format, for example. You'll likely learn to use the 'Cpanel' (control panel) provided by your web host to upload the HTML and image files to the hard drive that the web server software runs on. And perhaps along the way, you'll discover that you can add some interactive features to your web site by pasting in some PHP scripts to run on your server machine. And doing that may require creating a MySQL database to run some provided SQL code to create tables of data, where information generated by users of the web site is saved. And along the way, you'll likely learn to add some interactivity to the front end of your web site by pasting some pre-made JavaScript code into your HTML documents - and you'll likely need to edit this code in very simple specified ways, to personalize it for use on your site. During your interactions with a web server, you may likely need to perform some operations at the command line of the Linux OS which the web server runs on. It's common, for example, to have to run some commands which invoke a C compiler to build a Python interpreter (from the C language that Python was written in), and then to install some Python code packages using 'PIP' (the Python ecosystem's code library manager), so that you can run higher level services in Python code on your web server (for example, to perform artificial intelligence data mining on user data)... This whole routine is one common entry point into the lower layers of tech which allow 'average users' to perform commonly useful computing tasks in the modern world.
Another common entry point into layers of tech is through coding in spreadsheets and other office apps. To automate tasks which aren't enabled by built-in Excel functions, you may need write some 'VBA' (Visual Basic for Applications) script (code). And expanding from simple spreadsheet tasks, you may need to connect to a database system, and write some SQL code queries to get data back and forth between spreadsheet and database - using the spreadsheet as a front end report generating system for data which is stored on a network server. And to do all this, you'll likely need to have an understanding of how to use deeper features of the OS, its command line console, network concepts, etc.
Another entry point into layers of technology may come in the form of game play. Massively popular games such as Minecraft and Roblox enable interesting features through code which hobbyist gamers write, to automate and enhance the creation and interactive potential of their 3D worlds in those games. Lua is a common language in that domain. Of course, learning to write code in games involves learning more about the entire technological infrastructure in which it runs, and learning about this infrastructure enables further capabilities outside of games. Games also require fast machines with the most powerful components, so it's common for gamers to learn about hardware specs and to assemble their own PCs from specific parts, in order to own a computer with the best combination of capability and cost, for the games they prefer to play. Tens of millions of players have been exposed to these pieces of technology through games.
The control of IoT devices is another entry point into network tech and the layers of operating system use. 'Average' modern users may get drawn into understanding and using the lower levels of computing technology which allow those pieces of hardware to interoperate. Connecting a network of 'smart' TVs, Alexas and specialized hardware in a home, often requires digging deeper into those intersecting layers of common technology, and perhaps even editing/writing code to personalize/automate its use. Similarly, electronics hobbyists typically work with the raw guts of robotics equipment (hardware, OSes, programming languages, etc.), and children even learn to assemble hardware and write controlling code, with very popular toys such as Lego Mindstorms.
An interest in graphic design, as well as audio and video production, typically leads to lower level use of technology layers, since most often, those types of media are created and distributed using all sorts of tools that rely on every layer of modern tech (the Internet, web servers, scripting languages used to control editing and output, networked machines to share files, etc.).
Another common entry point to deeper technical understanding comes from setting up a local wireless network at an office, so that Word documents and printers can be shared between multiple users. This will likely involve learning a bit about network hardware (cards, routers, etc.) and protocols that enable data transfer between machines. Along the way you'll also need to learn how the OS provides access to files on hard drives on each of the connected machines - and to hook up the system, you'll need to understand IP addresses and client-server software concepts, and you may run (and even edit) some code scripts in an OS console window to install or automate certain features.
Really, no matter what you do for work or play, it's hard not to come in contact with lower levels of tech that make up all the tools we use every day. Even in our cars, entertainment systems and supermarket checkout lines - we're now interacting with computers, front and center, every day, everywhere. If you want to understand and master the basics of how all that modern technology works, start by learning how to use all the features and configuration options in the most popular operating systems (Windows, Mac, Linux, Android, iOS), and then learn the basics of a few popular programming languages (Python, Java, JavaScript, C#, C), and learn everything about how those language's surrounding ecosystems work. Learn to build a basic web site, how to assemble a desktop PC from components, how to share files and hardware on a local area network, and how to build an electronic project using a Raspberry Pi or Arduino microcontroller. Doing those things will expose you, in practical and useful ways, to how all modern technology works, and how you can *use and compose* the common fundamental layers to satisfy your own needs and interests. From there, you can research more about all the topics mentioned in this text, and build a solid understanding of how any new and emerging technology works, down to the lowest level of 1s and 0s being manipulated by metal hardware. The skills you develop by using and peeling apart the layers of any hardware platform, OS, user interface system, and programming language environment, will transfer, at least in useful concept, general workflow, and understanding, to other ecosystems (i.e., switching between C# on Microsoft Windows, or Objective-C on Mac will require learning new language and OS commands, but the general understanding and workflow will have direct corollaries). Understanding how those pieces are created, how to use them, and how they interact through common hardware designs, network technology, standardized file formats, established transfer protocols and popular server systems (such as http:// and all the pieces of the Web), and tools such as common database systems, SQL and programming languages, etc., is where your practical effort should be focused. Those are the established pieces and parts that everything in modern tech is built from.
This author recommends starting with Python as a first language, and Anvil as a first framework for building apps and web sites, but that is a purely personal choice. Python is among the easiest languages to learn if you haven't programmed before, it can be used to control virtually any sort of technology on most modern platforms, it's free, and is well known/supported in almost every environment, on just about every sort of device imaginable. Python's ecosystem enables you to easily import powerful libraries and all their dependencies, without much technical knowledge or time consuming work, and there is much less 'churn' (constant change in how versions and new tools work), than with JavaScript, for example.
JavaScript does have a tremendously popular ecosystem, and is at the heart of the 'native' tooling of web development, but the current state of volatility and rate of change in tooling, creates a need to learn and re-learn new tools regularly. JavaScript also exists within the structure of the web, which was not originally intended to host complex software. The complexity of working in a system that includes constantly evolving third party browser specifications, mixing code with HTML, CSS and frameworks that gain and lose popularity on a semiannual basis, sending user data over TCP and HTTP protocols, to a server which runs a potentially different OS, and code in a potentially different language than the front end, to be saved in one of many potential database systems - this huge mess tends to reduce productivity dramatically.
The Java language ecosystem and the Microsoft ecosystem (with C# and various flavors of Visual Basic languages) are two other incredibly popular platforms for development, which are well suited to enterprise environments. The tooling in these ecosystems has traditionally been 'heavy' (requiring huge IDE's and supporting software which can take hours to install, and requires powerful computers to run comfortably). Both communities support free and open source tools, backed by large companies. Be aware that an ethos of complexity is prevalent in communities which thrive in enterprise development work, so approaches to coding can tend to be more complex than absolutely necessary in these environments.
The ecosystem surrounding C/C++ is another enormous one, but it's best suited for system development, and perhaps game development (although using C++ is not nearly as important as it used to be in the field of game development).
Ruby on Rails, and the Ruby language have built up another popular ecosystem used in web and mobile development.
For building mobile apps, the native languages, tools, and ecosystems surrounding those platforms are still front and center (Java and Kotlin tools for Android, Swift and Objective-C for iOS). Whatever language(s) and platform(s) you gravitate towards, you will build useful skills, and be able to create productive technology. Take a look at each one, and choose the ones that make most sense to your brain and your interests.
With all that said, you can perform very productive work in almost any domain of interest, after only a few weeks of studying Python. Anvil is a Python framework that enables you to easily build web sites and apps that run on any device with a web browser, and to connect with code on any other device that can run Python code. Anvil has an easy to use WYSIWYG (what you see is what you get) GUI interface builder, a database system (Python ORM based on the powerful PostgreSQL RDBMS), and many common features such as simple user management (so users can log in with Google, Facebook, Microsoft, email, etc., to gain access to personalized features in apps), integrated credit card processing with Stripe, easy interactivity with JavaScript APIs (to make simple work of adding features such as hosted video conferencing, or any other third party online service), and many other common features needed to created fully working applications that enable practical capabilities in the real world, with the full power of the entire Python ecosystem, on the front end and back end, all in an incredibly easy-to-deploy hosted application environment which eliminates most of the troublesome time consuming work needed to deliver software to users (and the Anvil app server is free/open source, so your applications don't need to be hosted on Anvil's servers). Anvil's documentation gets new users started immediately creating useful applications, and covers an astonishing breadth of app types, from typical CRUD tasks (scheduling, email management, document creation) to full commercial e-commerce web site creation, and everything in between, all with a level of simplicity and powerful functionality that you won't find in many other systems. There are plenty of other tools which can be used to accomplish all the same ends. Anvil just provides a large scope of capabilities in one neatly integrated, consistently simple to use package, using a language that you can apply to many other fields of interest. There aren't many technological tools which can get you creating productive solutions as quickly as Anvil and Python - this can be a tremendous help if you're just starting out. Learning to actually do as many useful things with other programming language frameworks can take months to years. If you're just starting out, such a learning curving can be daunting enough to make you quit before ever learning to create a single useful app.
If you're particularly interested in creating games, start by learning to use Unity. You can create extraordinarily engaging and impressive apps with it very quickly, and the learning resources to help you ramp up to more complex games are better than with any other tool. Search 'learn Unity' on YouTube and Google, and if you're interested in creating games, your free time for the next few years will be fully occupied. You could also delve into scripting games such as Roblox with LUA, if you want to start with something a bit lighter.
If you're interested in building computer controlled electronics projects, a popular place to start is with a Raspberry Pico microcontroller kit. The cost for a Pico is less than $10, and you can get a starter kit full of sensors, displays, motors, breadboards, and everything else you need to build dozens of projects, for less than $50. The Raspberry Pi microcontrollers use 'MicroPython' as a programming language, which makes it easy to begin with, and if you learn Python to perform other tasks, the learning curve is reduced even further. Raspberry Pis are used in many commercial products, and there is an enormous amount of community and manufacturer support for them (endless tutorials, example projects, existing code to use, parts available, etc.). By starting down the 'maker' path with Raspberry Pi (or other products such as Arduino microcontrollers), you can learn to invent and implement smart mechanical and robotic hardware which can potentially perform any imaginable physical task. If you'd like to build a smart planter, for example, that senses sunlight and moves to give plants more or less light, and uses soil moisture sensors to automatically provide water when needed; or perhaps you'd like to set up motion sensors and a camera at your front door, and use artificial intelligence libraries to identify who is arriving at your door step, and then alert you with a text message, for example - projects like that can be accomplished by 'average' users with just a little understanding of modern technology, and such simple projects can serve as springboards to creating inventions with limitless possibilities. Take a look at https://www.youtube.com/watch?v=vsTTXYxydOE for a fantastic case study about how one developer created a robotic pool stick which can beat human players at billiards.
Do the work of building a small website, construct a simple electronic project/robot, make a little custom business app, or a mobile game, all with a single language in the beginning. Just follow along a few tutorials to start out - try the links below if you need somewhere to begin:
https://youtu.be/VchuKL44s6E
https://anvil.works/learn/tutorials (THESE are eye opening!)
https://makeuseof.com/micropython-raspberry-pi-pico/
Get your hands dirty interacting with, using, and composing some layers of programming language technology, to create something new or specialized for your needs. Diving into all that with one tool/ecosystem, is the best way initially. If your influences lean you towards starting with Java, C#, JavaScript, Ruby, Visual Basic, or any language other than Python and some other framework than Anvil, that's all fine (although it may take you a frustratingly long time to actually create any sort of productive app with other languages/ecosystems) - just dive in and learn to start creating pieces of technology with tools on some platform. Once you've begun, you'll begin to see how all the basic concepts are used repeatedly in every environment.
PREPARING FOR THE FUTURE, AND CONCLUSION
You will continue to see computing technology develop in ways that build higher and higher levels of complexity and human engagement, upon lower levels of existing technology. Everything in computing is still based upon binary bits being turned on and off at lightning speed within CPU, RAM, and microcontrollers within standardized hardware - although now there are billions of little transistors built into microscopic integrated circuits the size of a fingernail. Machines are still connected via decades-old network technology and protocols - although many of those connections are now passed through fast wireless and cellular hardware. Humans still interact with operating systems using graphical user interfaces - although mobile devices and even desktop machines have refined those interfaces to work best with touch screens, as opposed to mouse and keyboard inputs - and they are additionally amended with voice recognition and NLP (natural language processing) interfaces. Devices such as Alexa and some automobile systems actually rely entirely upon voice recognition and NLP for their human-OS interface, so things are certainly evolving in that regard. The Internet has very much replaced operating systems as the high level 'layer' upon which applications are developed, but keep in mind that every component which makes up the Internet (browsers, server software, programming language interpreters, etc.) all run on traditional operating systems, written in C, and which run on the same metal componentry (CPU, RAM, hard drive, microcontrollers, buses, etc.) that have been the foundation of computing for many decades - even if now they are often run as virtual machines on cloud servers (... but of course, those servers run on traditional metal components, OSes, legacy network protocols, etc...).
Hardware is getting smaller and prettier. Programming languages are improving to be more efficient and portable across platforms. Graphics, media, gaming and virtual reality systems are developing to create much more engaging entertainment and learning experiences. Video cards with fast dedicated GPUs (graphics processing units) are standard on even inexpensive mobile devices. Artificial intelligence is creating truly new capabilities that will genuinely change how computers can take part in our natural lives, and perform functions that are more human-like. Robotics and biological human-computer interfaces are providing a glimpse of future horizons which can only be imagined. But with all that, the foundations have all remained the same. Incredibly powerful artificial intelligence capabilities and the control of robots that are unimaginably complex internally, can be accomplished by simply importing an existing library and writing a few lines of code in Python - a language released in the 1990's.
If you want to really understand how all of computing works, study each of the terms and topics glanced over here - just perform a Google search on any term in this document and you'll go down a deep rabbit hole every time. This document is only meant to provide some minimal context, to introduce some basic concepts and terms, and to provide an overview about the most important topics in computing. Now go learn some Python or use any tool to build a web site - you'll fall down each of the necessary rabbit holes, as they come along.
ADDENDUM 1 - SOME NUMBERS RELATED TO MEMORY/STORAGE CAPACITY AND SPEED
Measuring RAM memory and hard drive storage space:
8 bits = 1 byte (1 character)
1024 bytes = 1 kilobyte
1024 kilobytes (KB) = 1 megabyte
1024 megabytes (MB) = 1 gigabyte
1024 gigabytes (GB) = 1 terabyte
1024 terabytes (TB) = 1 petabyte
The first IBM XT personal computer came with 64K (64 kilobytes, or 65,536 character) of RAM memory. Floppy disks ranged in size up to 1.44 MB (megabytes), and the first popular hard drives held about 10 MB of data (~10 million characters). These days, RAM memory is measured in GB (gigabytes), and a 5 TB (terabyte) external hard drive, which can connect immediately to any popular OS via a USB cable, costs about $100. Sizes for SSD (solid state drives) are a bit smaller, but they run faster, have fewer moving parts, and require less power. USB 'flash memory' sticks currently go up to 512 GB in size (for ~$50).
Processor Speeds:
1 KHz (kilohertz) = 1000 operations per second
1 MHz (megahertz) = 1000 KHz
1 GHZ (gigahertz) = 1000 MHz
The original IBM XT 8088 CPU ran at 4.7 Mhz (it could perform 4.7 million operations per second). Modern CPUs, even those found in mobile devices are measured in multiple gigahertz (they can perform billions of operations per second). The current Intel Core i9 runs at 5.5 GHz. The speed of a computer is not determined solely by it's processor clock speed. Early CPUs ran 8 bit architectures, and when the first Pentium CPUs came out, their clock speed was 75MHz, where the previous generation of 486 processors ran at 100MHz, but the 486 had a 32 bit architecture (it dealt with data in 32 bit word chunks), whereas the Pentium had a 64 bit bus, which meant it could process more data in a given amount of time, even at a slower clock speed (it had a wider data pipe). Most modern CPUs are 64 bit, and many GPUs (Graphics Processing Units, found on high end video cards) are 128 bit.
The speed equation doesn't end there. Modern CPUs now typically contain more than one core, which means that they can multitask and perform parallel processing tasks with far greater speed (when one core is completely busy, processing of other tasks can be performed on other cores). In order for multiple cores to be used to their best potential, the software running on them must be optimized to make use of parallel processing features. Other factors such as the speed of the RAM memory, the speed of the installed video card (especially in graphic intense games), how many applications are running, etc., can play a part in how quickly a machine 'performs' at any time. The design of the software which a computer runs, can also play a huge part in how quick it feels to the user. Running new software, and especially new operating systems, on old machines, can make a previously quick machine perform sluggishly, and vice-versa. Often a bottleneck in performance isn't even related to the machine - a slow Internet connection, for example, can make any network related activity feel slow, no matter how fast the computer can perform internal calculations. Heat (improperly seated heat sinks), broken fans, and bad air circulation caused by dusty environments can also slow down a computer.
ADDENDUM 2: MORE INTERNET RELATED HISTORY, TERMS AND NOTES
The network connection concepts and protocols that form the basis of the Internet were first devised by the military's ARPA program. The first public connections to ARPANET's 'wide area network' system were made available at large schools, government facilities, and businesses. Before the introduction of browser applications, other programs such as Archie and Gopher allowed users to search and download files on remote machines (they were command line precursors to search engines), and Telnet allowed users to log into a 'terminal' command line on other connected machines, to run remote programs (much like on old dial-in BBSs). It was the invention of the browser in 1990, that made the Internet friendly for average users. With this invention, commercial ISPs (Internet Service Providers) such as Compuserve and AOL, who had previously offered services similar to earlier local BBS connectivity, began offering dial-up modem (phone line) access to the Internet, including WWW web servers (which were first accessed with popular Mosaic, Netscape, and Internet Explorer browsers), and email servers, in the mid 1990s. By the early 2000s, new ISPs began offering 'broadband' (fast, always-on) DSL connections which ran over phone lines, 'cable' connections which transferred data on coaxial wires originally intended for television programming, and cellular phone networks originally created for voice communications. Those connections have progressed to include dedicated modern FIOS (optical) cable laid in the ground, and direct home satellite connectivity.
The role of companies offering ISP and media services largely flipped during the 2000s. The biggest cable TV, media and communications companies of the past (Comcast, Verizon, AT&T, and others) have become the most dominant modern ISPs. Now, media such as TV, as well as VOIP (voice over IP) and video chat communications are delivered more often by 'streaming' services, which send their content digitally over Internet connections. Companies such as Netflix, Hulu, Amazon, Disney+, YouTube, and TikTok aggregate and create media (old and new TV, movie, and video content), which can be accessed by software applications that run in PC browsers, in mobile apps, on smart TVs, and on devices such as Fire Stick, Roku, Apple TV, and Chromecast that plug into the HDMI ports of modern televisions. In the case of TV stick devices, each company's streaming software apps run inside the device, which connects to your home Internet WIFI connection, and outputs a signal to the TV screen. 'Smart' TVs simply have the apps for streaming services, as well as the WIFI hardware built in. Users can also choose to install an app from any streaming service onto a phone/tablet device, and send their screen output to a television using 'Smart View', 'AirPlay', or some other local wireless screen casting connection protocol. Users pay a monthly fee to access the content from any streaming company that owns shows they want to see, and then download (stream) that content in real time (while watching) from providers' back end services, using any device which runs that provider's client software. The constant availability of media, as well as the massive industry which has evolved surrounding new media content creation, is one of the largest driving business forces online today.
Apps such as Zoom, Facetime, Duo, Skype, Facebook Messenger, and WhatsApp enable users to connect via live video chat, to share desktops, and to transfer personal files directly with other people who are connected to the Internet. Since Covid, 'remote learning', 'remote work', and online socialization using these platforms have introduced significant lifestyle changes for many people.
The hardware that connects to an ISP is called a 'modem' (the phone modems of the 1990s worked as fast as 56K - they transferred up to 56,000 bits per second). Most modern modems that connect to 5G cellular, cable, FIOS, and satellite services, transfer tens to hundreds of millions of bits per second. When you connect to the Internet, the modem is the only device at your location which gets assigned a WAN (wide area network) IP address accessible on the Internet. The data traveling through a modem is dispersed to individual devices connected to the LAN (Local Area Network) in your home/business, by a 'router'. The modem and router provided by your ISP is typically combined into one physical device, and it provides WIFI (wireless local connections) to other devices (PCs, tablets, phones, home appliances), which all have the same standard WIFI hardware and network protocols installed.
The IP addresses of machines connected to a WIFI network are not exposed on the Internet. Those 'local' devices only connect to the local router, using a radio signal that travels just a few hundred feet. Local devices are most commonly assigned network IP numbers in the range 192.168.1.. Machines on a local area network can connect to external server IP addresses on the Internet, through the router, but machines on the Internet cannot connect directly to computers behind a local area network 'firewall', because local machines don't have IP addresses attached to the WAN Internet (only the router has a reachable WAN address). In order to run any server software (such as Apache, or an email server, for example) a computer inside a router's firewall, must have one of the 65,535 'ports' (network channels) 'forwarded' to the IP address of the specific machine running the server software. For example, web server software needs port :80 to be forwarded to the IP address of the computer running that software. You can read how to forward ports in your router's manual. Other types of network application (such as video chat), make their initial connections via third party servers, and then transfer data 'point to point' directly between users' routers, using protocols that send data through firewalls without the need for port forwarding.
In the past, before WIFI became commonplace, LAN connections were made via physical Cat5 cables. You can still find wired Network Cat6 jacks on modern laptops and desktop PCS, because they're useful in situations where data needs to be kept secure (i.e., not broadcast on WIFI radio waves, which can potentially be captured and decoded by nefarious users).
To be clear, outside your WIFI LAN, data signals to/from your ISP can be transferred over any sort of optical FiOS, coaxial cable, wireless 5G cellular signal (that connects to towers which are wired into the ground), or via wireless satellite communications. It doesn't matter which one of those methods your modem uses to connect to an ISP - any service available on the Internet, such as streaming TV, video chat, and all the websites, can be reached and used by any app within your local area network, via any type of WAN connection - as long as your ISP provides fast enough data, with a high enough total data transfer 'cap' to satisfy the bandwidth requirements of apps you want to use. Be aware that 'unlimited' cell plans typically have a high speed data cap of several dozen gigabytes, and then they drop down to much slower 2G connectivity, once that limit is reached each month. Also be aware that 'hot spot' (shared Internet access for other local devices, provided by a mobile phone) is typically limited to an even smaller monthly usage cap.
It's important to understand that apps used in a WIFI network can be written to run in a web browser, on a mobile device, and on dedicated pieces of hardware meant specifically for a certain purpose, such as media streaming or any sort of connected IoT appliance application. The pieces of the Internet all work together because they connect via *standardized protocols* at various levels: TCP and UDP network packet exchange protocols, HTTP:// data exchange protocol, JSON data format, standard document, image, and sound file types (HTML, PDF, JPG, MP3), the H.264 streaming video codec, etc. Those protocols standardize the connection routines, network package transmission, and data packaging formats used to transfer information. It's the standardization of these protocols, and the availability of standard network connection hardware on every sort of device, that enables all our modern technology to interact and share data, and to enable meaningful (or at least useful) human interactivity. We can expect this interactivity to become even more profoundly impactful as technologies such as self-driving cars and human body implants become pervasive. Artificial intelligence, virtual reality, social networking, media delivery and communication systems, IoT devices and robotic machines are already becoming more tightly woven into the fabric of our lives, and will continue to change how we think, and how we live.
ADDENDUM 3: WAYS IN WHICH PROGRAMMING LANGUAGE CODE IS RUN
There are literally thousands of programming languages available for just about every common operating system platform. When a developer writes code in a programming language, that code will generally be either 'interpreted' by an interpreter program which takes the human written code instructions as input, and performs the specified coded actions upon some specified data; or it can be 'compiled' to machine language which the computer runs natively on metal hardware.
Compiled languages tend to run (much) faster, because they create machine code which executes directly on the CPU, RAM, etc. C and other 'systems' development languages tend to be compiled, as they are most often used to create hardware drivers, operating system components, browsers, and other pieces of software upon which ever higher level software are built (web pages, utility scripts, etc.). C language code compiles directly to executable machine language (Windows .exe files, Linux and Mac .so files, etc.). Most modern factory-produced hardware also uses code written in C, to control operations. It has been described that using C to build user apps is like using a razor blade to mow a lawn. C provides very fine control of every operation, and can be optimized to squeeze every bit of performance out of hardware, but can be time consuming when used to build complex applications.
Interpreters require a separate intermediary program, usually running in an operating system, or within another even higher level piece of software (running on top of a game engine, for example). This real-time deciphering of human language commands and code structures, on the fly while the code is running, uses up many CPU cycles, RAM interactions, and other resources. Because of this, interpreted languages tend to run slower than languages compiled to machine code. HTML/CSS/JavaScript, Python, Ruby, Lua, and others that run within browsers, game systems, and other common human interactive environments, are typically interpreted (much like BASIC was in early home PCs).
It's also possible for languages to run on a VM ('virtual machine'), for which human language is partially compiled into code that's more like machine language, but which needs one more level of interpretation, at a more efficient lower level, closer to how the metal works. This provides the portability of interpretation, along with performance closer to that of compiled code. Java and C# are examples of languages that compile to VM code, using a VM 'interpreter' made to run on many different architectures, so that it's portable between many varieties of hardware and OS.
These days, interpreter speed doesn't cause as much of a critical slow down as it did decades ago, and when computationally intensive processes need to be sped up, libraries of C code and machine language can be accessed for computations that require efficient performance (i.e., for rigorous math calculations, to render complex game and VR graphics, to perform complicated AI processes (which rely on intense calculations by the CPU, RAM, etc.)). Connecting performant libraries with easy to use interpreted languages is common in today's high level coding practices. Interpreted languages are much easier to implement, and offer more malleable features within front-end web architecture, for example, but compiled code libraries may be called on the server side to perform complex computational work upon data submitted via a web page user interface.
When learning to create software on your computer, you'll discover several common work routines which developers use during the coding process. The simplest is the 'REPL' (read evaluate print loop). When you download and install an interpreted language such as Python, Ruby, or Lua, you can type the name of the language executable (i.e., python.exe) at the system command prompt (i.e., 'cmd' in Windows), and you'll enter into a new prompt which executes lines of code in the selected language (rather than OS commands). For example, if you install Python, and type 'python' at the OS console, you'll see the Python REPL run, where you can type individual lines of Python code. REPLs can be useful when learning all the built-in code constructs, and functions contained in libraries, which are commonly used in the language of your choice. But beyond learning, REPLs are helpful when working out little pieces of code which will be incorporated into larger systems (for example, run on a back-end server). You can experiment with and work out the syntax of function parameters, variable names, loops, data structures, and other fundamentally useful parts of your potential code, without the complexity of all the surrounding code in a large project. Take a look at https://replit.com/languages/python3 to see REPLs for most common programming languages, which run directly in any modern browser (so you can write/test code without having to install any language interpreter(s) on your computer).
The next level of work routine is to write multiple lines of programming language code into a 'source code' file. This works similarly to a REPL, except instead of executing each line individually, you'll type all the code lines into a text file, and edit the whole file as needed (using a regular text editor such as notepad.exe), before running the code. Then, when the code has been completed, you can go to your system command prompt, CD (change directory) into the folder containing your source code file (perhaps named something like 'myprogram.py', for a Python code file) and run it at the command line, using the language interpreter (i.e., you'll type something like 'python myprogram.py' to run the source code).
Most interpreter installers set up an OS 'path', so that the OS knows which folder on the hard drive to look in, to find the interpreter. For example, if the Python interpreter is installed in C:\python\, and your source code is in C:\mypythonfolder\, you'd need to type 'C:\python\python.exe C:\mypythonfolder\myprogram.py to run your code. With the OS path settings set to look in C:\python\, then you can CD into any folder on your hard drive, and just type 'python' plus the name of your source code file in that folder, and the OS will know where to look to run Python, without you having to type in the folder name where python.exe is stored. The language's installation routine will typically provide an option to set up your OS path variables automatically.
One other benefit that most interpreted language installers offer is to 'associate' the language filename extension (i.e., '.py'), so that any files saved with that extension will automatically be run through the language interpreter (i.e., python.exe for files ending in .py), whenever a user clicks on the file's icon in an OS file explorer app. For example, if you create a text file with Notepad, filled with Python code, and save it as 'myprogram.py', on your desktop, you can simply click on the myprogram.py icon on your desktop, and the OS will use the installed python.exe program (contained in the folder stored within OS path variables), to run that script. That enables a simple way to distribute programs written in any interpreted language - users of a program written in any language simply need to install the language, and then they can run code files in a way similar to native compiled applications (i.e., by just clicking on code file icons to run programs).
Compiled languages are implemented just a bit differently than interpreted languages. Developers write source code files, and then run a compiler program, which outputs an executable 'binary' file containing machine language instructions for the computer hardware and OS API. 'GCC' is a common free compiler which runs on most modern platforms. Microsoft Visual C++ is another that's used in MS Windows. On the command line, you can run 'GCC myprogram.c -o myprogram', for example, and GCC will output a myprogram.exe executable (or myprogram.so, depending upon your OS). The path to GCC and other compilers is typically set in OS variables, in the same way as it is for language interpreters. A compiled executable file then runs on its own, without the need for an interpreter to decipher the human language source. Making any changes to source code in a compiled language requires re-compiling the entire program again to a new executable. If your program is large, this process can take a long time.
Most programming languages actually provide very little functionality built-in. Languages definitions generally consist of code structure and syntax, some basic function definitions, as well as some fundamental features such as math, conditional evaluations, loops, etc. Most useful work is performed by higher-level functions in libraries. Many modern languages, for example, have functions which allow you to manipulate, search, and sort strings of text, but older languages such as C don't even have this capability built-in. Every language tends to have some sort of 'standard library', as well as some popular third party libraries which implement the most used functionalities needed to create software. Those libraries contain code which has been written, vetted, and improved upon by years of development work, so that users of the language never need to reinvent critically useful functionality on their own. Each language has its own method for including useful libraries of code - typically in the form of functions that perform useful actions upon data, and return a result, which gets assigned a variable label. The code to import libraries (or just specific parts of the libraries, or even just single functions from a library), is typically written on the first few lines of a source code file. This modular approach to including only the code needed in a project, helps to keep language systems lean and performant - it limits the need to import millions of lines of supporting library code into every app that doesn't use all the functionality in all those potential libraries.
Programming languages and libraries change over time, and often these changes break how code works. Using 'virtual environments' is one method of dealing with differences between versions of evolving languages and libraries. If you write code which works in version 3.10 of a given language, and version 5.7 of a library, for example, and your code no longer works in version 4.3 of the interpreter, or version 6.9 of the library, users will no longer be able to run your code without receiving an error. Virtual environments (and other 'versioning' systems) allow you to specify a particular version of interpreter and library to use, when running a particular code file, so that users don't have to figure out any of that.
Most developers don't write code in Notepad. Instead, they use IDEs which provide smart code completion, path and interpreter/library version control, as well as error and style checking ('linting'), which help to speed up the development process. If you're writing in a compiled language, IDEs include automated routines for selecting a compiler and libraries, running generated executable binary files, finding and fixing code errors, and going through the edit-compile-run process more efficiently.
These days, programming languages and libraries of functions are just the base pieces of development tooling used to create software. 'Frameworks' provide completely functional high level environments upon which applications can be built even more easily. For example, the Anvil framework consists of an entire web based IDE (which runs in any popular browser, so you can use it on any machine, without installing any software). The IDE provides WYSIWYG drag and drop user interface layout/design, as well as access to a wide variety of built-in service capabilities such as user management (so users of apps you create can log in securely using Google, Facebook, and Microsoft accounts, and you can easily grant access to features in their user profile, with just a few simple lines of code), e-commence credit card processing, simple visual database table construction/use (without having to install a database system or use any SQL code), simple interactivity with other machines that can run Python code (without having to write complex network socket connectivity code), deployment to Anvil servers with just the click of a button (without having to set up web server software, manually upload code files, etc.), and it provides seamless interaction between front end, back end, and database, all with one language and unified interface. Python is used as the language to enable these features in the Anvil framework, to assign variable labels to pieces of data and to perform if-then evaluations, loops, etc. But all the time-consuming traditional work required to code such high level features, is handled by the framework. The laborious work of installing interpreters, libraries, versioning systems, IDEs, web server software, database software, online accounts with CC processors, etc. - and then writing code to implement complex features in code, and *integrating* all those pieces on the front end and back end - typically all in different languages that are required for each piece (HTML/CSS/JavaScript in the browser, perhaps PHP on the web server, SQL in the database software, etc.), and using different tools to work with each of those pieces: remote terminal commands to configure a Linux OS machine which your web server runs on, a variety of IDEs to edit and test pieces of code on your local Windows/Mac machine, browser based file managers to upload finished source code files to your server, and a complex cycle of using all these tools to run and test new code before delivering updated versions of an app, without breaking currently running versions of code that function properly for users (but perhaps are just missing a few new features), while you make all those updates... that's all an extremely time consuming and complicated process to manage, before even thinking about how to write the code which implements any sort of useful capability in your system (and testing/integrating existing code/libraries to enable those capabilities).
The driving force behind the existence of so many new high-level frameworks is that they provide unified, simplified systems which integrate all the needed tools, so that with just a few lines of code, and perhaps some dragging and dropping, plus a few button clicks to deploy working software to users, you can implement seriously useful, high-level capabilities, without any of the work of building common code solutions and supporting infrastructure from the ground up. Frameworks such as Anvil even provide standardized layout/design choices, already implemented by default in the drag-drop UI system, which would have to be manually coded from scratch, if such a framework wasn't being used. Like other frameworks, Anvil allows you to edit built-in styles as deeply as you need, or to implement any library, or to use any existing lower level JavaScript code (which can connect to and implement even lower level C/machine language on the server, WASM in the browser, etc.), at any point in the development process. The purpose of any framework is to provide a powerful set of commonly used pieces of technology, with the most commonly expected design defaults already set, and drop-in functionality simply added, so that developers can implement and deploy complex features, to a variety of platforms, instantly and easily, and then modify and integrate specific features with low-level code, without having to deal with all of the messy disjunct pieces of architecture traditionally required to build applications from scratch.
Starting out with software/hardware development, and coding of any sort, is a far more friendly process than it was even a decade ago. There are millions of developers providing instruction, and developing tools to simplify coding work. A Google search for any common coding problem will yield immediate explanations and code solutions. High level frameworks and completely self contained ecosystems of toolkits for completing any sort of high-level work (3D games with Unity or Godot, web sites with Anvil or Ruby on Rails, electronic hardware projects with Raspberry Pi or Arduino, and thousands of others) reduce the learning curve from years to weeks. You can look up a video on YouTube to 'learn fast', and then dive into tutorials which explain/demonstrate exactly how to use a high-level framework in that language to create a useful application, in just a few days. You can learn step by step exactly how to finish and deploy a variety of projects within a matter of weeks, and become fluent in editing and even writing similar projects from scratch within a matter of months. With a learning curve that's similarly challenging to mastering Excel spreadsheets, you can now create truly unique and powerful software/hardware solutions that potentially effect your ability to achieve meaningful goals in business, hobbies, entertainment, and personal life. Starting out just requires getting over the initial hump of learning some basic language concepts and tooling operations. If you have an interest in technology, you can start for free in virtually any ecosystem, and scale up capability with only tiny initial investments in technology tools. The greatest investment is your time, and the learning process can be exciting and fun, if you choose the right tools. Try Python, Anvil, Raspberry Pi and other pieces of the Python ecosystem, if you want a small initial learning curve, free initial cost, and lots of support for any imaginable type of project, along with deeply capable, long term potential ability, potential for commercial work, and lots of fun along the way. Whatever technology you develop an interest in, use Google to search for answers to any questions you have!
ADDENDUM 4: THE NETWORK MODEL
(TODO: the network layers)