Lecture
This page contains explanations of some laws, principles, and patterns, but there is no advocacy in their favor. Whether to apply them or not is always a matter of debate, and everything depends on what you are working on.
The principle of least knowledge
The Law of Demeter (LoD) is a set of design guidelines for software development, particularly for object-oriented programs, that places constraints on the interactions between objects (modules).
In general terms, the Law of Demeter is a specific case of loose coupling. The rules were proposed in late 1987 at Northeastern University (Boston, Massachusetts, USA).
Put simply, every software module:
General statement of the rule: Object A must not be able to gain direct access to object C if object A has access to object B and object B has access to object C.
Thus, the code a.b.Method() violates the Law of Demeter, while the code a.Method() is correct.
Advantages
The advantages of the Law of Demeter are that code developed in accordance with it makes writing tests easier, and the resulting software is less complex to maintain and offers greater code reusability. Because objects are less dependent on the internal structure of other objects, object containers can be changed without modifying their callers (clients).
Disadvantages
A disadvantage of the Law of Demeter is that it sometimes requires the creation of a large number of small wrapper methods (delegates) to pass method calls through to internal components.
Amdahl's Law is a formula that demonstrates the potential speedup of a computational task that can be achieved by increasing a system's resources. It is typically used in parallel computing, and it can predict the actual benefits of increasing the number of processors given the limits on how parallelizable a program is.
Consider an example. If a program consists of two parts – part A, which must run on a single processor, and part B, which can be parallelized – it is clear that the benefits of adding more processors to the system running the program are limited. It can potentially speed up part B a great deal – but the speed of part A will not change.
The following diagram shows examples of potential speedup:
As you can see, even if 50% of a program can be parallelized, the benefits of adding more than 10 separate processors are negligible. If a program can be parallelized by 95%, improvements are noticeable even after adding a thousand processors.
As Moore's Law and the pace of processor speedups slow down, parallelization is becoming the key to improving efficiency. A great example is graphics programming – modern shader-based programming makes it possible to render fragments of an image in parallel, which is why modern graphics cards contain thousands of processing cores (GPU or shader units).
The broken windows theory holds that visible signs of crime (or a lack of care for the environment) lead to an increase in the number and severity of crimes (and to further deterioration of the environment).
This theory has been applied to software development, suggesting that poor code quality (or so-called "technical debt") can create a feeling that all attempts to improve quality will be ignored or undervalued, which in turn leads to the appearance of more bad code. This effect compounds cascadingly, so quality deteriorates over time.
Adding manpower to a late software project makes it later.
Brooks's Law states that in many cases, attempts to speed up the release of a project that is already late by adding more people to it result in the project being released even later than it otherwise would have been. Brooks, however, emphasizes that this is an oversimplification of the problem. His reasoning is as follows: given the time it takes to bring new resources up to speed and for people to communicate with one another, in the short term the development speed of the project drops. Also, many tasks may not be divisible, meaning they cannot easily be distributed among the additional resources, so the potential increase in speed turns out to be not as significant.
The common saying «nine women can't make a baby in one month» relates to Brooks's Law, in particular because some kinds of work cannot be divided or parallelized.
This is the central theme of the book «The Mythical Man-Month».
Conway's Law states that the technical constraints of a system being designed will reflect the structure of the organization. It is often invoked when trying to improve how an organization works. The law says that if an organization is structured as many small, unconnected modules, then the software it produces will be the same way. If an organization is built around verticals focused on particular features or services, then its software will reflect that fact.
The best way to get the right answer on the internet is not to ask a question, but to post the wrong answer.
Steven McGeady says that in the early 1980s Ward Cunningham gave him this advice: «The best way to get the right answer on the Internet is not to ask a question, but to post the wrong answer». McGeady named it «Cunningham's Law», though Cunningham himself denies authorship and says he is «misquoted». Although the phrase originally referred to interactions on Usenet, the law has since been used to describe how other communities work (Wikipedia, Reddit, Twitter, Facebook).
Dunbar's number is a limit on the number of stable social relationships a person can maintain. This refers to relationships that involve knowing the distinctive characteristics of a particular individual with whom the relationship must be maintained – their character, as well as their social standing and their connections to other people.
The exact number of such relationships is unknown. Dunbar himself suggested that a person can comfortably maintain no more than 150 such relationships. He described it in a more social context: «The number of people you would not feel embarrassed to join uninvited for a drink if you happened to run into them at a bar». Estimates of this number usually range from 100 to 250.
Just like stable relationships between people, maintaining a programmer's relationship with code requires effort. When faced with large and complex projects, or with ownership of many small ones, we rely on certain conventions, policies, and process models. Dunbar's number is important to consider not only as the number of employees grows, but also when determining the scope of a team's work, or the point at which a system should acquire supporting tools for modeling and automating logistics. In an engineering context, it is the number of projects (or the normalized complexity of a single project) for which you would confidently join the group maintaining the code.
The S-curve is a model that describes how a technical system (or technology) develops over time:
from emergence → through rapid growth → to saturation and slowdown.
The graph is usually plotted as follows:
The shape is an S (sigmoid).
The main stages of the S-curve
Example: the first computers, the first AI models (before the 2010s)
Example: the internet in the 2000s, deep learning after 2012
Example: classic internal combustion engines, traditional databases
Example: the limits of CPU frequency (the GHz race stopped)
The key idea of the law
No technology develops indefinitely
At some point, improvements become too expensive
example of the S-curve development of artificial intelligence systems
The reliability curve of technical systems is a classic model from reliability theory, often called the “bathtub curve”. It shows how the failure rate changes over time.
The essence of the law
The failure rate (λ) goes through three characteristic stages:
1) Initial period (infant mortality / burn-in)
Example: new electronics that fail in the first few days
2) Normal operation (the plateau)
Example: a server that runs reliably for years
3) Wear-out (aging of the system)
Example: an HDD begins to develop bad sectors, the mechanics wear out
The failure rate:
where:
Connection to other laws
This curve is often used together with:
A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system.
Gall's Law states that attempts to design a very complex system are almost certain to fail. Highly complex systems are rarely created in one go – they evolve from simpler ones.
A classic example is the World Wide Web. In its current state, it is a highly complex system. Originally, however, it was defined as a simple way to exchange content between institutions. It handled that purpose very successfully and evolved, over time becoming more complex.
Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.
It is also often formulated as:
When a measure becomes a target, it ceases to be a good measure.
Marilyn Strathern
Goodhart's Law (principle) concerns the use of metrics and states the following: «When a measure becomes a target, it ceases to be a good measure», because it becomes an object of manipulation, both direct (falsifying the numbers) and indirect (working solely to improve that measure). Thus, if an economic indicator becomes the objective function for conducting economic policy, the prior empirical regularities that use that indicator cease to hold.
There are many related concepts. For example, Donald Campbell noted that introducing indicators or criteria by which the performance of an institution is evaluated inevitably leads to a distortion of both the indicators themselves and the social processes they are meant to measure. The corresponding Campbell's Law appeared, in various formulations, as early as 1969.
K. Chrystal and P. Mizen held that the Lucas critique and Goodhart's Law are very close, and that Lucas has priority: although Goodhart's publication (1975) predates Lucas's (1976), the Lucas critique was presented at a conference in 1973 and was widely known before publication. The Lucas critique is usually applied in discussions of macroeconomic indicators, and Goodhart's Law — in discussions of monetary policy.
Chrystal and Mizen also found a connection with the physical uncertainty principle (measuring a system's parameters affects that system) and with T. Haavelmo's «invariance problem» (the relationships between economic quantities can change when external conditions change — just as an engineer who experimentally assessed a car's behavior on a flat, straight road would find that his formulas do not describe off-road driving, because quantities that previously did not change (invariants) begin to change).
John Danielsson formulated Goodhart's Law as «any statistical relationship will break down when used for policy purposes» and proposed a corollary for modeling financial risks: «A risk model breaks down when used for regulatory purposes».
Mario Biagioli linked Goodhart's concept to the consequences of using citation metrics to assess the importance of scientific publications, recalling that under Goodhart's Law people begin to game the characteristics of the economy that are used as indicators
The law states that optimizing on the basis of certain measures can devalue those measures. Overly selective metrics (KPIs), applied blindly to a process, lead to distortions. People strive to optimize the process locally, «gaming» the system in order to satisfy a particular metric, instead of paying attention to the global outcome of their actions.
Examples:
Never attribute to malice that which is adequately explained by stupidity.
The principle states that actions leading to a negative result may not have been carried out with bad intentions. A negative result is more likely explained by the fact that those actions and their consequences were not understood well enough.
It always takes longer than you expect, even when you take into account Hofstadter's Law.
You may have encountered mentions of this law while dealing with estimates of project timelines. In the field of software development, it seems a truism that we are not very good at accurately estimating how long it will take to complete a project.
A quote from the book "Gödel, Escher, Bach: an Eternal Golden Braid".
Improvement means deterioration.
This law states that improving one part of a system leads to the deterioration of other parts, or conceals other kinds of deterioration, which overall results in the system degrading compared to its current state.
For example, reducing the response time in a particular part of a system may lead to an increase in its throughput and, as a consequence, to capacity problems somewhere along the request path, which may affect another subsystem.
We tend to overestimate the effect of a technology in the short run and underestimate it in the long run.
The hype cycle – a visualization of the enthusiasm around and the development of a technology over time, originally created by Gartner. It is best illustrated by a graph:
After a technology appears, its popularity rises to the peak of inflated expectations, then plunges into the trough of disillusionment, climbs the slope of enlightenment, and reaches the plateau of productivity
In short, the cycle states that a fountain of enthusiasm usually arises around a new technology and its potential consequences. Teams often quickly become enamored with these technologies and often end up disappointed with the results they get. This may happen because the technology is not yet mature enough, or because the ways of applying it have not yet been thought through. After a certain amount of time, the capabilities of the technology grow, and the number of practical applications increases, after which teams can finally become productive.
With a sufficient number of users of an API, it does not matter what you promised in the contract: all observable behaviors of your system will be depended on by somebody.
Hyrum's Law postulates that if your API has enough users, then for any of the possible behaviors of your system (even one not described in the public contract) there will be a user who depends on it. A trivial example would be non-functional features of an API, such as response time. A more subtle example is consumers who rely on determining the type of an error by applying a regex to its description. Even if the public contract says nothing about the content of the message, and the implication is that users should use the error code, some of them may decide to use the message, and changing the message will break the API for those users.
Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.
Kernighan's Law is named after Brian Kernighan and is taken from a book he wrote with Plauger: «The Elements of Programming Style».
Everyone knows that debugging is twice as hard as writing the code. So if you put all your mental effort into writing the code, how are you going to debug it?
While the law is a hyperbole, it argues that it is better to use simple code than complex code, because debugging any problems that arise in complex code can turn out to be too costly or even impossible.
In network theory, the usefulness of a network grows roughly as the square of the number of its users.
The law is based on the number of possible pairwise connections within a system, and is closely related to Reed's Law. Odlyzko and others have argued that Reed's and Metcalfe's laws overstate the value of a system by not accounting for the limits of the human ability to comprehend a network; see Dunbar's number.
Metcalfe's Law states that the usefulness to a consumer of participating in a network is numerically measured as half the square of the number of network users (n2/2
). This estimate of usefulness is based on counting the number of direct connections in a network with n nodes
, which is mathematically expressed by the triangular number n(n−1)/2
, which asymptotically, for large n, tends to ∼n2/2
.
It is assumed that a single direct connection in a network potentially brings a participant 1 conventional unit of benefit. Then with 5 participants the benefit will be 10 (by the formula n(n−1)/2) conventional units, for a network of 10 participants it will already be 50 units, and so on.
This law was first formulated by Robert Metcalfe in 1980, though not from the standpoint of users, but rather from the standpoint of «compatible communicating devices» (telephones, fax machines). After an article in Forbes on September 13, 1993, the law came to be associated with Ethernet users.
The law is used to numerically describe the network effect, including in multi-level network marketing, in social networks, and other network structures.
Two telephones allow only one interaction, five create 10 connections, and with twelve there are 66 connections
Graph of Metcalfe's Law
Swanson's Law is an empirical law which states that the price of solar photovoltaic cells tends to fall by 20% for every doubling of cumulative volume. At present, prices are halving roughly every 10 years. The law is named after Richard Swanson, the founder of SunPower Corporation, a manufacturer of solar panels.
Swanson's Law is compared to Moore's Law, which predicts the growth of the computing power of processors. The prices of crystalline silicon photovoltaic cells fell from $76.67 per watt in 1977 to $0.36 per watt in 2014. The change in the module price in US dollars over time amounts to a decline of 10% per year.
Swanson's Law
The father of Stephen Stigler, the economist George Stigler, studied the history of economic discoveries. He said: «If an earlier, valid statement of a theory falls on deaf ears, and a later restatement is accepted by the science, this is surely proof that the science accepts ideas only when they fit into the then-current state of the science». He also gave some examples where a discoverer did not receive due recognition.
Robert Merton used the term the Matthew effect to describe a pattern in which a well-known scientist takes precedence over a lesser-known one in the matter of being credited with authorship. And even if their results were similar, credit for the discovery is usually attached to the already famous scientist. Merton wrote: «This pattern of recognition, skewed in favor of the established scientist, appears principally (i) in cases of collaboration and (ii) in cases of independent multiple discoveries made by scientists of distinctly different rank».
Boyer's Law was formulated by Hubert Kennedy in 1972. It states: «Mathematical formulas and theorems are usually not named after their original discoverers», and it was named after Carl Boyer, whose book A History of Mathematics contained many examples of this pattern. Like Stigler, Kennedy noted that it is a rare instance of a law whose statement confirms its own validity.
The saying «Everything of importance has been said before by somebody who did not discover it» is attributed to Alfred Whitehead.
In Russia, Stigler's Law is often called «Arnold's principle»; V. I. Arnold formulated it in his popular-science note of 1998 and described in detail the situation in which this principle was formulated:
The English physicist Michael Berry named this eponymic principle «Arnold's principle», supplementing it with a second one. Berry's principle: Arnold's principle applies to itself (that is, it was known earlier as well). I had communicated the eponymic principle to him in response to a preprint about the «Berry phase», examples of which, in no way inferior to the general theory, had been published decades before Berry by S. M. Rytov (under the name «inertia of the polarization direction») and A. Yu. Ishlinsky (under the name «the drift of a submarine's gyroscope due to the mismatch between the path of return to base and the path of departure from it»)
Schneier held that peer review and expert analysis are very important for protecting cryptographic systems. Mathematical cryptography is usually not the weakest link in the security chain. Therefore, effective protection requires that cryptography be combined with other things.
The term Schneier's Law was coined by Cory Doctorow in his talk about Digital Rights Management for Microsoft Research. The law is formulated as follows:
Any person can invent a security system so clever that he or she cannot imagine a way to break it.
The intelligence explosion is a possible consequence of the creation and development by humans of strong artificial intelligence, described by Irving Good in 1965.
According to this concept, at some point humanity will create an artificial intelligence capable of improving itself, and doing so without human involvement. Thus, in the process of self-improvement, such an artificial intelligence could transform into an artificial superintelligence that would surpass human abilities. This would be the moment of the «intelligence explosion». The concept of the intelligence explosion is closely related to the concept of the technological singularity.
The number of transistors placed on an integrated circuit chip doubles approximately every 24 months.
Moore's prediction, which is often used to demonstrate the enormous rate of improvement in semiconductor and chip manufacturing technologies, turned out to be surprisingly accurate and held from the 1970s to the end of the 2000s. In recent years the trend has changed somewhat, in particular because of the physical limits on miniaturizing components. However, progress in parallelization and potentially revolutionary changes in semiconductor technology and quantum computers may mean that Moore's Law could remain valid for the next several decades.
Anything that can go wrong will go wrong.
Murphy's Law, attributed to Edward A. Murphy, postulates that anything that can go wrong will inevitably go wrong.
This saying is often used by developers. Sometimes unexpected things happen during development, testing, or even in production. It can be connected to a law more commonly used in Britain, Sod's Law [which, in fact, is also known in Russia / translator's note]:
If something can go wrong, it will, and at the worst possible moment.
Usually these «laws» are used in a humorous sense. However, phenomena such as confirmation bias and selection bias can lead people to become overly enamored with these laws (in most cases, when everything works as it should, nobody notices, whereas failures are more noticeable and attract more discussion).
Entities should not be multiplied without necessity.
Occam's Razor states that of several possible solutions, the most likely will be the one that contains the fewest concepts and assumptions. This solution will be the simplest and will solve only the problem at hand, without introducing incidental complexities and possible negative consequences.
Work expands to fill the time allotted for it.
In its original context, the law was based on a study of bureaucracy. Pessimistically, it can be applied to software development by assuming that a team will work inefficiently until the project's deadline begins to approach, and then will rush to deliver it on time, which makes the specific completion date rather arbitrary.
If you combine it with Hofstadter's Law, you get an even more pessimistic view: work will expand until it fills all the time needed to complete it, and it will still take longer than expected.
Premature optimization is the root of all evil.
In Donald Knuth's work «Structured Programming with go to Statements» he wrote: «Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%».
However, premature optimization can also be described as an attempt to optimize something before we understand what we need to do.
Technology is dominated by two types of people: those who understand what they do not manage, and those who manage what they do not understand.
Every technical hierarchy, in time, develops a competence inversion.
These statements suggest that, because of various selection criteria and the tendencies of how groups are organized, there will always be a number of experienced people at the working levels of a technical organization, and at the management level there will always be people who have no idea about the complexities and problems of the work they manage.
It should be emphasized, however, that laws like this are a very crude generalization, and may be applicable to some types of organizations and not applicable to others.
The usefulness of large networks, particularly social networks, scales exponentially with the size of the network.
This law is based on graph theory, where usefulness scales as the number of possible subgroups, which grows faster than the number of participants or possible pairwise connections. Odlyzko and others have argued that Reed's and Metcalfe's laws overstate the value of a system by not accounting for the limits of the human ability to comprehend a network; see Dunbar's number.
The law states that a system has a certain amount of complexity that cannot be reduced.
Part of a system's complexity may arise unintentionally. It is the result of poor structure, errors, or a flawed model of the problem being solved. Unintentional complexity can be reduced or eliminated. However, some kinds of complexity are an inherent consequence of the complexity of the problem itself. This complexity can be moved, but not eliminated.
One interesting element of this law is the assumption that even when the whole system is simplified, its inherent complexity is not reduced but rather shifted onto the user, who is forced to behave in a more complicated way.
All non-trivial abstractions, to some degree, are leaky.
This law states that abstractions, which are usually used in IT to simplify working with complex systems, in certain situations «leak», letting through elements of the systems underlying them, which causes the abstraction to start behaving unpredictably.
An example is loading a file and reading its contents. The file system API is an abstraction over lower-level kernel systems, which are themselves an abstraction over the physical processes involved in changing data on a magnetic platter (or in the flash memory of an SSD). In most cases, the abstraction that represents a file as a stream of binary data will work. However, sequential reading of data from a magnetic disk will be faster than random access to it, whereas an SSD will not have such problems. You need to understand the details lying deep down in order to handle these cases (for example, database index files are structured to reduce random-access time) when the abstraction «leaks» the implementation details a developer needs to be aware of.
The given example can become more complicated with the addition of new abstractions. The Linux OS makes it possible to access files over the network, yet they appear locally as «ordinary» ones. This abstraction will «leak» in the event of network problems. If a developer treats them as «normal», without taking into account that they are subject to latency problems and network failures, his solution will be suboptimal and buggy.
The article describing the law suggests that an excessive fondness for abstractions, combined with a poor understanding of the processes underlying them, in some cases even complicates the process of solving the problem.
Examples: Photoshop starting slowly. I ran into this problem in the past. Photoshop started up very slowly, sometimes taking several minutes. Apparently, the problem was that on startup it reads information about the current default printer. But if that printer was a network printer, this could take an extremely long time. The abstraction that a network printer is equivalent to a local one caused problems for the user in the case of a poor connection.
The law states that groups spend far more time and attention discussing cosmetic or trivial problems than serious and far-reaching ones.
The usual example given is the work of a committee approving plans for a nuclear power plant, which spends most of its time discussing the structure of the bicycle shed rather than the more important questions of the plant's design itself. It can be hard to make a valuable contribution to a discussion of very large and complex topics without extensive knowledge of the subject behind you. Yet people want to be recognized for making valuable observations. Hence the tendency to focus on small details that are easy to reason about but not necessarily important for the project as a whole.
The fictional example given above gave rise to the term «bikeshedding», describing the waste of time on discussing trivial details. There is a similar term, «yak shaving», describing seemingly project-unrelated activity that is part of a long chain of necessary preparatory steps.
The Unix philosophy is that software components should be small and focused on doing one specific task well. This makes it easier to build systems by assembling small, simple, and well-defined modules, instead of using large, complex, multi-purpose programs.
Modern practices, such as «microservice architecture», can be considered an application of this philosophy – the services are small and focused on performing one specific task, which makes it possible to compose complex behavior from simple building blocks.
An approach to team structure and organization promoted by Spotify. Under this model, teams are organized around product features rather than around technologies.
The model also promotes the concepts of tribes, guilds, chapters – other components of the organizational structure.
In the design of any language, the total amount of time spent discussing a feature in this list is proportional to two raised to the power of the feature's position in the list.
0. Semantics.
1. Syntax.
2. Lexical syntax.
3. Lexical syntax of comments.
That is, for every hour spent on semantics, 8 hours are spent on the syntax of comments.
Like the law of triviality, Wadler's Law postulates that in language design the amount of time spent on the constructs of a language is disproportionately large compared to the importance of those constructs.
Don't be a jerk.
This concise, simple, and all-encompassing law, formulated by Wil Wheaton, is aimed at increasing harmony and respect within a professional organization. It can be applied in conversations with colleagues, when conducting code reviews, when looking for objections to other points of view, in criticism, and in general, in the professional interaction of people.
Netscape developer Jamie Zawinski once observed: "Every program attempts to expand until it can read mail. Those programs which cannot so expand are replaced by ones which can."
This joke was funny because it hinted at programs that were originally not intended to work with mail clients. The humor faded, however, when it became apparent that many apps on Google Play request access to phone components and user data that have nothing to do with their core tasks.
Some saw this as an attempt to track users, but perhaps it also reflects the human tendency to do something simply because it is technically possible.
Laurence Peter became famous for stating that in a hierarchy a person rises to a position for which they are utterly incompetent. But he also managed to say something about complex projects:
«A complex project will become too complex to be understood even by its own developers».
In other words, if part or all of the software differs greatly from what the user is accustomed to, it is best to change the design. Ideally, one should aim for incremental improvements that are large enough to make an impression but small enough to remain familiar to the user.
states:
"The first 90% of the code accounts for the first 90% of the development time. The remaining 10% of the code accounts for the other 90% of the development time."
This formulation, attributed to Tom Cargill, perfectly captures the paradox of development: completing a project always turns out to be harder and to take longer than initially assumed. Even experienced developers run into surprises, especially in the final stages, when small details require significant effort.
This idea is complemented by Hofstadter's Law:
"It always takes longer than you expect, even when you take into account Hofstadter's Law."
In real life, this rule is familiar to anyone who has launched a commercial project: you always have to spend twice as much time and money as planned to achieve only a fraction of the planned profit.
This is especially relevant for Ubuntu and Fedora developers, who work on releasing new versions every six months. Perhaps they should revisit these laws before every development cycle
states: there is always one more bug.
Historically, the first "bug" in a computer system was literal — a moth stuck in one of the relays of the Mark II computer caused it to malfunction.
Transferring this metaphor to modern technology, one can argue that no matter how thoroughly an operating system or program is tested, there always remains an unnoticed flaw that is discovered when it is already too late. This is an inevitable reality familiar to every computer user.
Grosch's Law — «the additional economy gained is only the square root of the increase in speed — that is, to make computations 10 times cheaper, you must make them 100 times faster». This observation about computer performance was made by Herb Grosch in 1965. The saying is more often formulated as follows:
Computer performance increases as the square of the cost. If computer A costs twice as much as computer B, then you should expect computer A to be four times faster than computer B.
The law can also be interpreted to mean that computers represent an example of economies of scale: the more expensive a computer is, the linearly better its performance-to-price ratio. This means that inexpensive computers cannot compete in the market, because their computations are more expensive. In the end, a few enormous machines would serve the computing needs of the entire world. Presumably, this could have been the motivation for Thomas J. Watson's prediction at the time that the total global market for computing tasks amounted to five computers.
A modern example: this law states that to have a computer a hundred times more powerful than a modern PC, the owner would have to pay only ten times more.
For clusters, the original Grosch's Law implies that if a cluster contains 50 machines and another 50 machines are added to it (doubling the cost), the resulting 100-machine cluster will have quadrupled computing power, which is obviously incorrect. On the contrary, even a linear increase in performance — making a 100-machine cluster twice as powerful as a 50-machine cluster — will be a challenge.
When Google was deciding on the architecture of the computer system for its web search service, it concluded that scaling up clusters of large and medium computers (mainframes) as the business grew would be too expensive; and so it chose cheap processors and disks for its computing arrays[
Van Loon's Law is stated as «The degree of technical development will always be in inverse proportion to the number of slaves at a country's disposal». The law is a poorly formulated statement made in Stuart Chase's book "Men and Machines", published in 1929. The author may be Hendrik Willem van Loon
Named after Linus Torvalds, the creator of the Linux kernel.
According to Eric Raymond, Linus's Law states that «given enough eyeballs, all bugs are shallow»; or, more formally, «given a large enough beta-tester and co-developer base, almost every problem will be characterized quickly and the fix will be obvious to someone». Raymond formulated this rule in the fourth part of his essay «The Cathedral and the Bazaar».
This is referred to as a «law of human nature», though Kelly's evidence is anecdotal.
The Machine Intelligence Research Institute released a document with a detailed description of a much larger set of AI predictions: 95 predictions extracted from a database of 257 AI predictions, which finds a wide range of estimates significantly before and after the assumed lifespan of the predictor, thus contradicting the law. MIRI states a stricter «Maes-Garreau» rule: «The predictor expects AI to develop precisely at the end of their life».
Clarke's three laws were formulated by the well-known English inventor, science-fiction writer, and futurologist Arthur Clarke:
The first of the three laws, originally called simply «Clarke's law», was formulated by the author in his book «Profiles of the Future» (1962). The statement now called the second law was also first mentioned in that edition, but Clarke called it the «second law» only in a later revision of his book in 1973. In that same edition Clarke formulated the third law, which is today the best known of all. In a subsequent revision of the same book, made in 1999, Clarke added a fourth law: «For every expert there is an equal and opposite expert» (literally «… an equal and opposite expert», which is an allusion to Newton's third law).
Isaac Asimov formulated an addition to Clarke's first law, concerning non-professionals' disagreement with a scientist:
«Nevertheless, when the lay public rallies round an idea that is denounced by distinguished but elderly scientists and supports that idea with great fervor and emotion — the distinguished but elderly scientists are then, after all, probably right».
The Three Laws of Robotics in science fiction are mandatory rules of behavior for robots, first formulated by Isaac Asimov in the story «Runaround» (1942).
Asimov's laws of robotics are built by analogy with Kant's categorical imperative and represent the most well-known attempt to create ethical rules for robots. By the author's design, the Three Laws are meant to impose an ethical cast on the behavior of a machine.
The laws state:
- A robot may not injure a human being or, through inaction, allow a human being to come to harm.
- A robot must obey all orders given to it by human beings, except where such orders would conflict with the First Law.
- A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.
Asimov's cycle of robot stories is devoted to the Three Laws, as well as to the possible causes and consequences of their violation. In some of them, on the contrary, the unforeseen consequences of robots obeying the Three Laws are examined (for example, «Mirror Image»).
In one of the stories in the cycle, an Asimov character comes to a conclusion about the ethical basis of the Three Laws: «…if you stop to think of it, the Three Laws of Robotics coincide with the basic principles of most of the ethical systems existing on Earth… put simply, if Byerley follows all the Laws of Robotics, he is either a robot, or a very good man».
In 1986, in the novel «Robots and Empire», Asimov proposed the Zeroth Law:
0. A robot may not harm humanity or, through inaction, allow humanity to come to harm.
The Three Laws of Robotics are a unifying theme for all of Asimov's fiction, including not only the robot cycle but also some of his other works.
Asimov's Three Laws of Robotics, right up to the 21st century, constitute the most well-known attempt to create ethical rules for robots.
Since 2014, with the emergence of prospects for the real-world application of artificial intelligence in engineering, active ethical debates have been under way around lethal autonomous weapons (combat robots), and Asimov's laws are used in these debates. In addition to Asimov's own works, the discussions draw on ethical foundations that stem directly from Kant's categorical imperative, from which Asimov, too, derived his laws.
The development of AI is a business, and business, as is well known, is not interested in developing fundamental safety measures — especially philosophical ones. Here are a few examples: the tobacco industry, the automobile industry, the nuclear industry. For none of them was it said from the outset that serious safety measures were necessary, and all of them resisted externally imposed constraints, and none of them adopted an absolute edict against harming people.
Principles are most often associated with advice on designing programs.
Companies tend to promote incompetent employees to management in order to remove them from the workflow.
A management concept developed by Scott Adams (the creator of the Dilbert comics), inspired by the Peter Principle. According to the Dilbert Principle, employees who could not be considered competent are promoted to management in order to limit the potential damage to the company. Adams first explained this principle in an article for the Wall Street Journal in 1995, and then described it in detail in his 1996 book «The Dilbert Principle».
For the most part, everything in life is distributed unevenly.
The Pareto Principle states that in some cases the smaller part of the inputs accounts for the larger part of the results:
In the 1940s, the American engineer of Romanian origin Joseph Juran, often called the father of quality management, began applying the Pareto Principle to quality problems.
This principle is also known as the 80/20 rule, the law of the vital few, and the principle of factor sparsity.
Examples: in 2002 Microsoft reported that after fixing the 20% most frequently occurring bugs, 80% of the related problems and crashes in Windows and Office would be fixed.
In a hierarchical system, every individual tends to rise to their level of incompetence.
A management concept created by Laurence Johnston Peter, it notes the fact that people who do their job well are promoted until they reach a level at which they can no longer cope (their «level of incompetence»). Because they have climbed high enough, they are less likely to be fired (unless they do something utterly foolish), so they will remain in that position for which they lack the necessary skills, since their skills at operating within an organization do not necessarily coincide with the skills required to succeed in the given role.
This principle is especially interesting to engineers, who begin their careers in purely technical roles but often build careers that lead to managing other engineers – which requires a completely different set of skills.
Be conservative in what you do, and liberal in what you accept from others.
This principle is often applied to the development of server applications. According to it, the data you send to others should be as small as possible and conform as closely as possible to the standard, yet you yourself should accept as input data that are not entirely standardized, if you are able to process them.
The goal of the principle – the creation of robust systems capable of digesting poorly formed data whose meaning can nonetheless be understood. However, accepting non-standard data as input can have consequences related to a breach of security, especially if the acceptance of such data has not been well tested.
Over time, the practice of accepting non-standard data can cause protocols to stop evolving, as those who implement data exchange begin to rely on the leniency of programs when creating new features.
An acronym denoting the following 5 principles:
S: The Single Responsibility Principle
O: The Open/Closed Principle
L: The Liskov Substitution Principle
I: The Interface Segregation Principle
D: The Dependency Inversion Principle
They represent the key principles of object-oriented programming. Design principles like these are meant to help developers create systems that are easier to maintain.
Every object should have a single responsibility, and that responsibility should be entirely encapsulated in the class.
The first of the SOLID principles. The principle states that every module or class should do only one thing. In practice this means that a small, single change to a program's functionality should require a change in only one component. For example, to change the procedure that checks a password for complexity, the program needs to be modified in just one place.
In theory, this makes code more robust and easier to change. The fact that the component being changed has a single responsibility should mean that it will be easier to test that change. Changing the password-complexity-checking component from the previous example should affect only the functions related to password complexity. It is much harder to reason about what a change to a component with many responsibilities will affect.
Software entities should be open for extension, but closed for modification.
The second of the SOLID principles. The principle states that entities (classes, modules, functions, etc.) should allow their behavior to be extended, but should do so in such a way that their current behavior cannot be modified.
A hypothetical example: imagine a module capable of turning a document in Markdown markup into a document in HTML markup. If the module can be extended so that it learns to handle new features of the Markdown format without changing its internal functions, then the module is open for extension. If the module's handling of current Markdown features cannot be changed, then the module is closed for modification.
The principle is especially closely related to object-oriented programming, where you can design objects that are easy to extend, but where you should not design objects whose internals will change in unexpected ways.
It should be possible to replace a type with a subtype without breaking the system.
The third of the SOLID principles. The principle states that if a component depends on a type, then it should be possible to use subtypes of that type such that the system does not refuse to work or require the details of that subtype.
For example, we have a method that reads an XML document from a structure representing a file. If the method uses the base type file, then it should be possible to use anything derived from a file in the function. If file supports backward seeking, and the XML parser uses that feature, but the derived type «network file» refuses to work with backward seeking, then «network file» violates this principle.
The principle is especially closely related to object-oriented programming, where type hierarchies need to be modeled very carefully in order to avoid confusion for the user of the system.
Software entities should not depend on methods they do not use.
The fourth of the SOLID principles. The principle states that the consumers of a component should not depend on functions of the component that it does not use.
For example, we have a method that reads an XML document from a structure representing a file. It only needs to read bytes, moving forward or backward through the file. If this method has to be updated because of changes to an unrelated feature of the file structure (for example, because of an update to the access-control model representing the file's security), then this principle will be violated. It is better for the file to implement a «seekable stream» interface, and for the XML method to use it.
The principle is especially closely related to object-oriented programming, where interfaces, hierarchies, and abstract types are used to minimize coupling between components. This principle forces the use of "duck typing", a methodology that eliminates explicit interfaces.
Higher-level modules should not depend on lower-level modules.
The fifth of the SOLID principles. The principle states that higher-level controlling components should not know the implementation details of their dependencies.
For example, we have a program that reads metadata from a website. Presumably, its main component would need to know about the component that downloads the web page's content, and the component that reads the metadata. If we take the Dependency Inversion Principle into account, then the main component will depend only on an abstract component that obtains byte data, which in turn depends on an abstract component capable of reading metadata from a byte stream. The main component will know nothing about TCP/IP, HTTP, HTML, and so on.
The principle is fairly complex, since it inverts the expected dependency in a system. In practice it also means that a separate controlling component must ensure the correct implementation of the abstract types (in the previous example, something must supply the metadata-reading module with a component for downloading the file over HTTP and for reading data from the HTML meta tag).
Every piece of knowledge must have a single, unambiguous, and authoritative representation within a system.
The Don't repeat yourself principle, or DRY, helps developers reduce the repetition of code and keep information in one place. It was mentioned in 1999 by Andy Hunt and Dave Thomas in their book «The Pragmatic Programmer».
The opposite of the DRY principle would be the WET principle – «Write Everything Twice» or «We Enjoy Typing».
In practice, if the same information is duplicated in two or more places, use the DRY principle by merging them into one place and reusing it as needed.
Keep it simple, stupid
The KISS principle says that most systems work best if they are kept simple; consequently, simplicity should be a key goal in design, and unnecessary complexity should be avoided. It originated in the US Navy in 1960, and the phrase is attributed to the aircraft designer Clarence Johnson.
It is best illustrated by the example in which Johnson gave a team of design engineers a small set of tools and instructed them to design an aircraft such that it could be repaired in the field in combat conditions by an average mechanic using only that set. Here «stupid» refers to the relationship between how things break and the complexity of the tools available to repair them, not to the intellectual abilities of the engineers.
An acronym for You Ain't Gonna Need It.
Always implement features only when you actually need them, and not when you think you might need them in the future.
Ron Jeffries, the author of the extreme programming (XP) technique and the book «Extreme Programming Installed», suggests that developers should implement only such functionality as is needed right now, and should not try to predict the future by implementing functionality that may be needed later.
Following this principle should reduce the amount of unused code in the codebase, as well as the effort and time spent on functionality that provides no benefit.
Landauer's Principle is a principle formulated in 1961 by Rolf Landauer (IBM), which states that in any computing system, regardless of its physical implementation, the loss of 1 bit of information releases heat Q:
where kB — the Boltzmann constant, T — the absolute temperature of the computing system.
The Shannon—von Neumann—Landauer (SNL) expression denotes the minimum energy Ebit required to process 1 bit (or — the minimum barrier height needed to separate two states of an electron ESNL):
At T = 300 K the energy ESNL ≈ 0.017 eV ≈ 2.7×10−21 J.
Although the increase in entropy from erasing a single bit is extremely small, modern microchips contain billions of transistors switching at frequencies of up to several gigahertz (billions of times per second), which raises the amount of heat from erasing information to measurable magnitudes.
At the beginning of the 21st century, computers, when processing a single bit, dissipated roughly a million times more heat than the principle predicts. However, by the early 2010s the difference had dropped to a few thousand, and it is predicted that it will continue to approach the Landauer limit over the coming decades.
The limits imposed by Landauer's Principle can be circumvented by implementing reversible computing, though this increases the requirements for memory capacity and the number of computations. It is also sometimes suggested that reversible computing will be slower.
Although Landauer's Principle is recognized as a physical law, it still requires experimental verification at various levels.
The universality of the principle was criticized in the works of Earman and Norton (1998), and then Shenker (2000) and again Norton (2004, 2011), and was defended by P. Bennett (2003) and Ladyman (2007)
In 2016, researchers from the University of Perugia claimed to have demonstrated a direct violation of Landauer's Principle, but according to Laszlo Kish, their results are erroneous, because they ignore the main source of energy dissipation, namely – the charging energy of the capacitance of the input electrode.
In 2018, the validity of Landauer's Principle was confirmed at the quantum level; in the experiment it was recorded that erasing the quantum information of a quantum computer's qubits also produces heat.
In 2020, it was shown that quantum effects can lead to a 30-fold increase in energy dissipation compared to the Landauer limit.
Also known as the fallacies of networked computing. This is a list of assumptions concerning distributed computing that can lead to software failures. These are the following assumptions:
The first four were listed by Bill Joy and Tom Lyon in 1991, and James Gosling first classified them as the «Fallacies of Networked Computing». Peter Deutsch added the 5th, 6th, and 7th fallacies. In the late 1990s Gosling added the 8th.
The group of engineers was inspired by the processes taking place at Sun Microsystems at the time.
These fallacies are worth carefully taking into account when developing reliable code. Each of the fallacies can lead to flawed logic incapable of coping with the reality and complexity of distributed systems.
Comments