41. Linear Programming

Nature, and all of nature, human being, life, works with certain constraint at any given time, and consciously, subconsciously, unconsciously, tries to find the best or most optimal path forward from that situation. And life is a series of such path forwards, each moment a first step with available constraints, reality and exercise of will, and available resources to the optimal outcome.

One is more objective when one considers life like this perhaps. One's own as well as everyone and everything around. Eventually, the reckoning is only with the self, and the infinite inside, that whether you are training yourself enough to see well, to keep solving your linear programs optimally.

It somehow traverses the ground between mathematics, programming, life as in available reality at any point in time, and a self with its own shifting objective function - it is one of the most fascinating concepts in terms of philosophy/maths and very accessible to any being inherently, unconsciously. Perhaps more accessible to the pre-word subconscious than the shallow knowledge confused being.

Its use is mostly in operations planning in the real world, and only perhaps so because in the strange way art mimics life, because all life's underlying operations are managed through this function.

Mathematically it is fascinating because it uses maths for life in some way, somehow channels the abstraction to a more material purpose. Overall more contenment in each sphere.

A step towards win-win outcomes. The best possible solution to it.

*

Personal takes aside, Linear Programming: (from various sources)

Linear programming is the technique used for optimizing a particular scenario. Using linear programming provides us with the best possible outcome in a given situation. It uses all the available resources in a manner such that they produce the optimum result.

*

"Linear programming (LP), also called linear optimization, is a method to achieve the best outcome (such as maximum profit or lowest cost) in a mathematical model whose requirements and objective are represented by linear relationships. Linear programming is a special case of mathematical programming (also known as mathematical optimization).

More formally, linear programming is a technique for the optimization of a linear objective function, subject to linear equality and linear inequality constraints. Its feasible region is a convex polytope, which is a set defined as the intersection of finitely many half spaces, each of which is defined by a linear inequality. Its objective function is a real-valued affine (linear) function defined on this polytope. A linear programming algorithm finds a point in the polytope where this function has the largest (or smallest) value if such a point exists."

*

"Linear programming or Linear optimization is a technique that helps us to find the optimum solution for a given problem, an optimum solution is a solution that is the best possible outcome of a given particular problem.

In simple terms, it is the method to find out how to do something in the best possible way. With limited resources, you need to do the optimum utilization of resources and achieve the best possible result in a particular objective such as least cost, highest margin, or least time. 

The situation that requires a search for the best values of the variables subject to certain constraints is where we use linear programming problems. These situations cannot be handled by the usual calculus and numerical techniques."

*

"In operations research, linear programming (LP) is one of the mathematical techniques used to get an optimal solution to a given operational problem, considering resource scarcity and external and internal constraints.

To apply Linear Programming for process optimization, these requirements have to be met:

Problem statement: define the objective in clear mathematical terms

Decision variables: quantitative input variables impacting the objective

Constraints: quantitative and measurable conditions

Objective function: the relationship between the objective and the input variables has to be linear"

In terms of modern applications apart from the philosophical contemplation of life, nature and their operations:

Linear programming is a widely used field of optimization for several reasons. Many practical problems in operations research can be expressed as linear programming problems.[6] Certain special cases of linear programming, such as network flow problems and multicommodity flow problems, are considered important enough to have much research on specialized algorithms. A number of algorithms for other types of optimization problems work by solving linear programming problems as sub-problems. Historically, ideas from linear programming have inspired many of the central concepts of optimization theory, such as duality, decomposition, and the importance of convexity and its generalizations. Likewise, linear programming was heavily used in the early formation of microeconomics, and it is currently utilized in company management, such as planning, production, transportation, and technology. Although the modern management issues are ever-changing, most companies would like to maximize profits and minimize costs with limited resources. 

*

An optimal solution need not exist, for two reasons. First, if the constraints are inconsistent, then no feasible solution exists: For instance, the constraints x ≥ 2 and x ≤ 1 cannot be satisfied jointly; in this case, we say that the LP is infeasible.

Certain realities about the basic of fabric of cosmos - the equity of the planes, the either end of equation transference indifference. If it is not met, there is no conversation, understanding - it is all horizontal plane randomness. No enduring meaning can ever appear.

*

Linear programming, a powerful mathematical technique, is used to solve optimization problems in various industries. Here are some modern applications:

Supply Chain Optimization: Linear programming helps companies minimize costs and maximize efficiency in their supply chains. It’s used for determining the most cost-effective transportation routes, warehouse operations, and inventory management strategies.

Energy Management: In the energy sector, linear programming is utilized to optimize the mix of energy production methods. This includes balancing traditional energy sources with renewable ones to reduce costs and environmental impact while meeting demand.

Telecommunications Network Design: Linear programming aids in designing efficient telecommunications networks. It helps in allocating bandwidth, designing network layouts, and optimizing the flow of data to ensure high-speed communication at lower costs.

Financial Planning: Businesses and financial analysts use linear programming for portfolio optimization, risk management, and capital budgeting. It helps in making investment decisions that maximize returns while minimizing risk.

Healthcare Logistics: In healthcare, linear programming is applied to optimize the allocation of resources, such as hospital beds, medical staff, and equipment. It’s crucial for improving patient care, reducing wait times, and managing costs effectively.

Manufacturing Process Optimization: Linear programming is used to determine the optimal production levels for multiple products within a manufacturing facility, considering constraints like labor, materials, and machine availability.

Agricultural Planning: Farmers and agricultural planners use linear programming to decide on crop selection, land use, and resource allocation to maximize yields and profits while conserving resources.

Airline Crew Scheduling: Airlines employ linear programming to schedule crews efficiently, ensuring that flights are staffed in compliance with regulations and minimizing operational costs.

*

Can be solved through iterative process of Simplex method, or through graphing, to viusally chart the optimal area.  (For the straightforward Linear programming Problems. There are several branching thoughts and variations on linear programming - as a wikipedia dive reveals).

"Simplex Method

Optimization Algorithm: The Simplex Method is a powerful algorithm used in linear programming to find the optimal solution to linear inequalities.

Step-by-Step Approach: It iteratively moves towards the best solution by navigating the edges of the feasible region defined by constraints.

Efficiency: Known for its efficiency in solving large-scale linear programming problems.

Versatility: Applicable in various domains like diet planning, network flows, production scheduling, and more, showcasing its versatility.

*

One of the main challenges is to formulate your problem correctly and accurately, which requires a clear understanding of your operations, your constraints, your objectives, and your assumptions.

 *

The main benefit of optimization models is the ability to evaluate possible solutions in a quick, safe, and inexpensive way without actually constructing and experimenting with them. Other benefits are as  follows.

1. Structures the thought process. Constructing an optimization model of a problem forces a decision maker to think through the problem in a concise, organized fashion. The decision maker determines what factors he or she controls; that is, what the decision variables are. The decision maker specifies how the solution will be evaluated (the objective function). Finally, the decision maker describes the decision environment (the constraints). Modeling acts as a way of organizing and clarifying the problem.

2. Increases objectivity. Mathematical models are more objective since all assumptions and criteria are clearly specified. Although models reflect the experiences and biases of those who construct them, these biases can be identified by outside observers. By using a model as a point of reference, the parties can focus their discussion and disagreements on its assumptions and components. Once the model is agreed on, people tend to live by the results.

3. Makes complex problems more tractable. Many problems in managing an organization are large and complex and deal with subtle, but significant, interrelationships among organizational units. For example, in determining the optimal amounts of various products to ship from geographically dispersed warehouses to geographically dispersed customers and the routes that should be taken, the human mind cannot make the billions of simultaneous tradeoffs that are necessary. In these cases, the decision maker often uses simple rules of thumb, which can result in less than optimal solutions. Optimization models make it easier to solve complex organization-wide problems.

4. Make problems amenable to mathematical and computer solution. By representing a real problem as a mathematical model, we use mathematical solution and analysis techniques and computers in a way that is not otherwise possible.

5. Facilitates “what if” analysis. Mathematical models make it relatively easy to find the optimal solution for a specific model and scenario. They also make “what if” analysis easy. With “what if” analysis, we recognize that the prices, demands, and product availabilities assumed in constructing the model are simply estimates and may differ in practice. Therefore, we want to know how the optimal solution changes as the value of these parameters vary from the original estimates. That is, we want to know how sensitive the optimal solution is to the assumptions of the model. “What if” analysis is also called sensitivity or parametric analysis.

Although mathematical modeling has many advantages, there are also disadvantages. The actual formulation or construction of the model is the most crucial step in mathematical modeling. Since the problems tend to be very complex, it is possible to mismodel the real problem. Important decision variables or relationships may be omitted or the model may be inappropriate for the situation. The optimal solution to the wrong problem is of no value.

A second disadvantage is not understanding the role of modeling in the decision-making process. The optimal solution for a model is not necessarily the optimal solution for the real problem. Mathematical models are tools to help us make good decisions. However, they are not the only factor that should go into the final decision. Sometimes the model only evaluates solutions with regard to quantitative criteria. In these cases qualitative factors must also be considered when making the final decision. The bottom line for evaluating a model is whether or not it helps a decision maker identify and implement better solutions. The model should increase the decision maker’s confidence in the decision and the willingness to implement it.

38. Markov Chains

There are certain events which are independent of each other, like subsequent coin tosses and their results, throw of dice. But there are others where the next event or state is dependent on current state.

Sort of current state changes the probability of next state. Such sequences are called Markov chain. 

Markov chains, named after Andrey Markov, are mathematical systems that hop from one "state" (a situation or set of values) to another. For example, if you made a Markov chain model of a baby's behavior, you might include "playing," "eating", "sleeping," and "crying" as states, which together with other behaviors could form a 'state space': a list of all possible states. In addition, on top of the state space, a Markov chain tells you the probabilitiy of hopping, or "transitioning," from one state to any other state---e.g., the chance that a baby currently playing will fall asleep in the next five minutes without crying first.

In independent chains, there is no 'memory'. In markov chains, there is memory in the system, or perhaps the system trains, adapts, learns.

Markov chain is conditioned on only one current state. 

A Markov chain has short-term memory, it only remembers where you are now and where you want to go next.

This means the path you took to reach a particular state doesn’t impact the likelihood of moving to another state. The only thing that can influence the likelihood of going from one state to the other is the state you are currently in.

 

Sequences of repetitions such as these where it is assumed that:

• the probability of each possible outcome is conditional only on the immediately preceding outcome, and
• the conditional probabilities for each possible outcome are the same on each occasion (that is the same matrix is used for each transition)

are called Markov chains, named after a Russian mathematician

In general, a Markov chain is defined by:

• a transition matrix, T, which for a situation that has m outcomes or states is a square matrix of dimension m × m. The elements of the transition matrix are conditional probabilities.
• an initial state vector, S0, which has dimension m × 1, and gives information about the Markov chain at the beginning of the sequence, or step 0. The elements of the initial state vector may be numbers, percentages or the results of an individual trial.

Here this article is very helpful to understand the basics and construction.

The next thing you do is to encode the dependencies between states, using conditional probabilities. In the context of Markov models, these conditional probabilities are called transition probabilities. Transition probabilities describe the transition between states in the chain as a conditional probability.

With two states (A and B) in our state space, there are 4 possible transitions (not 2, because a state can transition back into itself). If we're at 'A' we could transition to 'B' or stay at 'A'. If we're at 'B' we could transition to 'A' or stay at 'B'. In this two state diagram, the probability of transitioning from any state to any other state is 0.5.

Of course, real modelers don't always draw out Markov chain diagrams. Instead they use a "transition matrix" to tally the transition probabilities. Every state in the state space is included once as a row and again as a column, and each cell in the matrix tells you the probability of transitioning from its row's state to its column's state. So, in the matrix, the cells do the same job that the arrows do in the diagram.

A characteristic of what is called a regular Markov chain is that, over a large enough number of iterations, all transition probabilities will converge to a value and remain unchanged[5]. This means that, after a sufficient number of iterations, the likelihood of ending up in any given state of the chain is the same, regardless of where you start.

So, when the chain reaches this point, we can say the transition probabilities reached a steady-state.

This is what Markov ultimately wanted to prove!

Only regular Markov chains converge over time. And if your Markov Chain does not converge, it has a periodic pattern.

In Markov chains that have periodicity, instead of settling on a steady-state value for the likelihood of ending in a given state, you’ll get the same transition probabilities from time to time.

This other place also explains it visually. And other ideas.

A Markov chain is a stochastic model that uses mathematics to predict the probability of a sequence of events occurring based on the most recent event. A common example of a Markov chain in action is the way Google predicts the next word in your sentence based on your previous entry within Gmail. 

A Markov chain is a stochastic model created by Andrey Markov that outlines the probability associated with a sequence of events occurring based on the state in the previous event. It’s a very common and easy-to-understand model that’s frequently used in industries that deal with sequential data such as finance. Even Google’s page rank algorithm, which determines what links to show first in its search engine, is a type of Markov chain. Through mathematics, this model uses our observations to predict future events.

The main goal of the Markov process is to identify the probability of transitioning from one state to another. One of the primary appeals to Markov is that the future state of a stochastic variable is only dependent on its present state. An informal definition of a stochastic variable is described as a variable whose values depend on the outcomes of random occurrences.

37. Wrap Up (rivers, sugar)

Here's world's top 20 longest rivers. A better infographic:

And yet, one has to remember:

"(It's not so easy to define how long a river is. If a number of tributaries merge to form a larger river, how would you define where the river actually begins? Here is how we are defining river length:

River lengths or river-length data are affected not only by some of the natural and artificial causes noted in the preceding paragraph, but also by the precision of various techniques of measurement, by the scale of available maps or aerial photographs, and by somewhat arbitrary decisions. For example, the length may be considered to be the distance from the mouth to the most distant headwater source (irrespective of stream name) or from the mouth to the headwaters of the stream commonly identified as the source stream. "

Following are the river basins:

--

Here perhaps a look at dams across these rivers.

"In order to meet the needs of a growing population, a number of infrastructure projects (e.g., dams) were built to increase water withdrawals from rivers and groundwater for much of the 20th century. Specifically, during the second half of the 20th century, construction of dams was regarded as the main measure of river basin management for generating hydropower, controlling flood, storing water storage, and reducing risks from natural disasters"

By the year 2010, a total of 32,473 large dams had been constructed all over the world [27]. The number of large dams (as well as related reservoirs and hydropower stations) for each country is calculated and the result is shown in Figure 1 (see the blue circles). At the continental scale, there were 14,719, 9241, 5163, 1799, 988, and 563 large dams in Asia, North America, Europe, Africa, South America, and Oceania, respectively. It is worth noting that nearly 90% (i.e., (14,719 + 9241+5163)/32,473) of the large dams were built in Asia, North America, and Europe, which are the three continents with more developed countries than other continents. In terms of the locations, most large dams have been constructed in the basins of the great rivers, such as the Yangtze River, the Yellow River, and the Ganges River in Asia, the Mississippi River in North America, and the Rhine River and Danube River in Europe. Furthermore, at the national scale, it is observed that United States has built the most large dams with the number of 7968, followed by China (with the number of 4928) and India (with the number of 4104). Among them, the United States is the most-developed country in the world, while China and India are the two fastest developing countries. Therefore, such results can preliminary indicate the close relationship between large dams and economic development.

Moreover, based on the main use of a large dam given by Global Reservoir and Dam database [28], it is observed that 29% of the large dams were built for hydroelectricity, 34% for irrigation, 10% for flood control, 16% for water supply, and 11% for other uses (e.g., navigation and recreation). It is observed that hydroelectricity and irrigation are the two most important considerations for large dam construction for the purposes of energy production and food production, respectively.

---

Wrapping up - Sugar

Sugar subsidies have driven market costs for sugar well below the cost of production. As of 2019, 3/4 of world sugar production is never traded on the open market. Brazil controls half the global market, paying the most ($2.5 billion per year) in subsidies to its sugar industry

Following is slightly dated data on Sweetners.

 Since Sugar became expensive, ... Coke replaced it with corn related

In terms of end use, following is an indicative mix of direct-indirect sugar usage, and then details in one particular country, Germany.  (It is 20 year old data, directional reference only)

Different kinds of sugars:

34, 35 & 36. Sugar

VOLUME

The world produces 180 million tonnes of Sugar annually. (For comparison, the world produces ~800 million tonnes of wheat annually). And sugar is not the only sweetner. 

Brazil produces 23% of the world's sugar ~43 million tonnes, followed by India at 19% with 35 million tonnes.

In terms of consumption, India is the highest consumer of sugar on overall tonnage (32 million tonnes), next is Brazil with 16.5 million tonnes. Though on a per capita basis, some of the developed world counts as the highest sugar consumption daily average.

Per Capita Consumption

A couple of directional data on per capita consumption. Different data, perhaps not easily measured, based on the food retailed perhaps - more a factor of availability. Just as a directional indicator

International Trade in Sugar - Volume

Following table indicates the trading patterns in Sugar. Given the high production, Brazil is the largest exporter of sugar globally with 34.5 million tonnes exported (out of its total production of 45 million tonnes). Indonesia, China are largest importers importing ~5-5.5 million tonnes annually.

--

Sugar Cane Production

Sugar production is foreseen to expand due to, among other things, the flexibility of sugar mills to shift between sugar and ethanol production, which reduces the investment risks. Sugarcane accounts for around 86% of the sugar crops and sugar beet makes up for the remainder. Sugarcane is a perennial crop that grows mainly in the tropical and sub-tropical regions. The same plants can be harvested for several years, although yields decline over time. In addition to sugar and ethanol, sugarcane can also be used to produce derivatives such as electricity (through bagasse surplus) and bioplastics. However, it remains a water-intensive crop. Conversely, sugar beet is an annual crop, cultivated mostly in temperate zones. This crop is used to produce a wide range of products, including food (sugar), feed, bio-based products for the industry (pharmaceuticals, plastics, textiles, and chemicals), and ethanol. Over the outlook period, the increase in the production of sugarcane is foreseen to come from higher yields and area expansion. Brazil will continue to be the main producer of sugar and sugarcane-based ethanol, producing 39% of the world's sugarcane by 2029. This sugarcane will be used for 18% of global sugar production and 90% of global sugarcane-based ethanol production (compared to 17% and 91% during the base period).

Brazil produces ~782 million tonnes of sugarcane, and India 490 million tonnes.  (This number used to be 59 million tonnes in 1961 for Brazil, and 110 million tonnes for India in 1961. In 1961, the world produced 51 million tonnes of Sugar against current 180 million tonnes)

In terms of sugarcane production though, Brazil is significantly higher since a lot of sugarcane there is used for ethanol.

--

BRAZIL & Sugar

A deeper look at the largest producer, Brazil. 

Brazil - 29.4% of total land is crop and pasture land.  (21.1% pasture land, 8.2% agricultural land).

Sugar cane land represents 1.2% of Brazil's territory.

Sugar cane is one of the most important crops in Brazil, accounting for more than 10 % of the national agricultural production value in 2022.

The state of São Paulo is the largest producer, accounting for around half of the Brazilian sugar cane output. Minas Gerais and Goiás follow, ranking second and third, respectively. 

Sugar cane is used for Sugar and Ethanol. Additionally, sugar cane juice is used to produce cachaça, a distilled spirit which is the main ingredient in Brazil's famous Caipirinha.

History of Sugar in Brazil

Still, trying to understand the significant growth of Brazil in 2000s in the sugar industry, it seems that it has been so throughout for other agricultural commodities as well for Brazil during that decade.

Brazil is the largest country in terms of arable land, a top-5 producer of 34 agricultural commodities, and the largest agricultural net exporter. Its size and standing as a major supplier of commodities around the world and competitiveness in commodity markets suggest potential for continued growth in the agricultural sector. 

Since the mid-2000s, Brazil has accelerated its transformation from an exporter of mainly tropical agricultural products such as coffee, sugar, citrus, and cacao to a major global supplier of commodities, including soybeans, grains, cotton, ethanol, and meats.

Key factors:

Factors driving Brazil’s transformation include agricultural research that has increased yields, expansion of the arable land base, large investments in production technologies to develop crop and forage varieties, and increased global demand for food and animal feed, particularly over the last decade. Most important, Brazil’s ability to harvest two to three crops a year in the same plot of land makes it unique compared with other grain and soybean-producing countries. Other factors include export-oriented macroeconomic policies, extended periods of depreciation for Brazil’s currency (the real), crop-specific agricultural policy incentives, improved sanitary controls, acquisition of foreign competitors, and a growing multinational presence and foreign investment in the country.

And hence:

Further, the push for ethanol pushed the sugarcane industry in Brazil:

--

Sugar Prices

Even though sugar prices have risen in recent years, they have had a long-term downward trend and are very volatile. The relatively low cost of producing sugar in the leading exporting countries is one reason why sugar is cheap. Sugar is also frequently traded at less than the costs of production. 

Sugar is grown by millions of smallscale farmers. However, its supply is controlled by a huge and complex web of transnational millers, refiners, traders and processors with operations across the world.

Sugar price

Sugar cost of production

If you compare the two charts above, one can see that perhaps only Brazil has cost of production lower than the sugar prices.

Perhaps that's why:

"The United States makes about nine million tons of sugar annually, ranking it sixth in global production. The United States sugar industry receives as much as $4 billion in annual subsidies in the form of price supports, guaranteed crop loans, tariffs and regulated imports of foreign sugar, which by some estimates is about half the price per pound of domestic sugar." (2019)

--

History

The history of ‘White Gold’ is a reminder that even the mundane things in our kitchen cupboards, snuck into our food, and passing through the tedious stages of government quota consultations, are deeply tied up in the threads of exploitation that run throughout our food system – those of the past and the ones we’re still untangling today.

Sugar, which has become a necessary commodity almost has been one of the drivers of history.

None of this — the extraordinary mass commodification of sugar, its economic might and outsize impact on the American diet and health — was in any way foreordained, or even predictable, when Christopher Columbus made his second voyage across the Atlantic Ocean in 1493, bringing sugar-cane stalks with him from the Spanish Canary Islands. In Europe at that time, refined sugar was a luxury product, the backbreaking toil and dangerous labor required in its manufacture an insuperable barrier to production in anything approaching bulk. It seems reasonable to imagine that it might have remained so if it weren’t for the establishment of an enormous market in enslaved laborers who had no way to opt out of the treacherous work.

For thousands of years, cane was a heavy and unwieldy crop that had to be cut by hand and immediately ground to release the juice inside, lest it spoil within a day or two. Even before harvest time, rows had to be dug, stalks planted and plentiful wood chopped as fuel for boiling the liquid and reducing it to crystals and molasses. From the earliest traces of cane domestication on the Pacific island of New Guinea 10,000 years ago to its island-hopping advance to ancient India in 350 B.C., sugar was locally consumed and very labor-intensive. It remained little more than an exotic spice, medicinal glaze or sweetener for elite palates.

It was the introduction of sugar slavery in the New World that changed everything. “The true Age of Sugar had begun — and it was doing more to reshape the world than any ruler, empire or war had ever done,” Marc Aronson and Marina Budhos write in their 2010 book, “Sugar Changed the World.” Over the four centuries that followed Columbus’s arrival, on the mainlands of Central and South America in Mexico, Guyana and Brazil as well as on the sugar islands of the West Indies — Cuba, Barbados and Jamaica, among others — countless indigenous lives were destroyed and nearly 11 million Africans were enslaved, just counting those who survived the Middle Passage.

“White gold” drove trade in goods and people, fueled the wealth of European nations and, for the British in particular, shored up the financing of their North American colonies. “There was direct trade among the colonies and between the colonies and Europe, but much of the Atlantic trade was triangular: enslaved people from Africa; sugar from the West Indies and Brazil; money and manufactures from Europe,” writes the Harvard historian Walter Johnson in his 1999 book, “Soul by Soul: Life Inside the Antebellum Slave Market.” “People were traded along the bottom of the triangle; profits would stick at the top.”

 

Here's a historical look at sugar prices:

This saccharine decadence fell out of style among the British elite. But sugar consumption soon took off anyway with the rise in popularity of bitter drinks like coffee and tea. Sweetened with sugar and increasingly available, they entered the daily routines of a growing fraction of British society, which could consume them in the burgeoning number of coffee houses popping up across London.

To satisfy rising demand, European powers sent African slaves to their West Indian colonies, extracting 12 million tonnes of sugar—and destroying countless human lives—between 1690 and 1790. This was the infamous ‘triangle trade,’ where Europeans sent manufactured goods to Africa, slaves were sent from Africa to the West Indies, and the West Indies sent commodities like sugar to Europe.

Disagreements over sugar taxation in this web of trade were an important driver of tensions between Britain and its American colonies that ultimately led to war.

During the American Civil War, domestic sugar supplies crashed as slaves abandoned Louisiana sugar plantations. With over one tenth of the world’s sugar supply taken offline due to the war, and the failure of the French beet sugar crop, prices spiked from 7¢/lb in 1861 to over 20¢ in 1864.

While the institution of slavery was destroyed by the Union army in the US, it persisted in Cuba. But Cuban slaves’ fortunes soon turned, as a plantation owner fed up with Spanish colonial rule freed his slaves and revolted, starting a war that would last ten years and result in the emancipation of slaves over the coming decades. In the spring of 1869, speculators took advantage of this upheaval by attempting to corner the sugar market, predicting that the conflict would decimate the animals used to harvest and process Cuban sugar.

[[//Then there are other areas as well affected by sugar and colonial pursuits. After slavery was abolished, indenture systems came in place to provide cheap labour. This led to displacement, emigration - 

The Indian indenture system was a system of indentured servitude, by which more than 1.6 million workers[1] from British India were transported to labour in European colonies as a substitute for slave labour, following the abolition of the trade in the early 19th century. The system expanded after the abolition of slavery in the British Empire in 1833,[2] in the French colonies in 1848, and in the Dutch Empire in 1863. British Indian indentureship lasted until the 1920s. This resulted in the development of a large South Asian diaspora in the Caribbean,[3] Natal (South Africa), Réunion, Mauritius, and Fiji, as well as the growth of Indo-South African, Indo-Caribbean, Indo-Mauritian and Indo-Fijian populations.

On 18 January 1826, the Government of the French Indian Ocean island of Réunion laid down terms for the introduction of Indian labourers to the colony. Each man was required to appear before a magistrate and declare that he was going voluntarily. This agreement is known as girmit[6] and it outlined a period of five years labour in the colonies with pay of 8 rupees per month (about $4 in 1826) and rations, provided labourers had been transported from Pondicherry and Karaikal.

The Indian indenture system was put in place initially at the behest of sugar planters in colonial territories, who hoped the system would provide reliable cheap labour similar to the conditions under slavery.[7] The new system was expected to demonstrate the superiority of "free" over slave labour in the production of tropical products for imperial markets.

For more, perhaps VS Naipaul.]]

--

Plantation System

The geographic center of sugar cane cultivation shifted gradually across the world over a span of 3,000 years from India to Persia, along the Mediterranean to the islands near the coast of Africa and then the Americas, before shifting back across the globe to Indonesia. A whole new kind of agriculture was invented to produce sugar – the so-called Plantation System. In it, colonists planted large acreages of single crops which could be shipped long distances and sold at a profit in Europe. To maximize the productivity and profitability of these plantations, slaves or indentured servants were imported to maintain and harvest the labor-intensive crops. Sugar cane was the first to be grown in this system, but many others followed including coffee, cotton, cocoa, tobacco, tea, rubber, and most recently oil palm.

The people in New Guinea were among the most inventive agriculturalists the world has known. They domesticated a broad range of local plant species including not only sugar cane but also taro, bananas, yam, and breadfruit.

--

A related discussion is sugar and general health. A few notes:

What helps in terms of policies (basis various public health policies across different countries). It works by influencing:

-how available sugar and sugary products are - reducing availability in places like school canteen, retail environments, and in food supply by reformualtion

- how affordable they are - for example, soda taxes 

- how acceptable sugar and its alternatives are perceived to be - making water a preferred drink. Free fruit/veg a couple of times in shcools

- how aware we are of sugar in products  - packaging clearly, front label

--

Open - market size, key players, other sweetners. Islands affected.

33. Normal Distribution

To understand Normal Distribution, one first needs to look at Central Limit Theorem.

Central Limit Theorem:

In probability theory, the central limit theorem (CLT) states that the distribution of a sample will approximate a normal distribution (i.e., a bell curve) as the sample size becomes larger, regardless of the population's actual distribution shape.

Put another way, CLT is a statistical premise that, given a sufficiently large sample size from a population with a finite level of variance, the mean of all sampled variables from the same population will be approximately equal to the mean of the whole population. Furthermore, these samples will approximate a normal distribution, with their variances being approximately equal to the variance of the population as the sample size gets larger, according to the law of large numbers.

As a general rule, sample sizes of 30 or more are typically deemed sufficient for the CLT to hold, meaning that the distribution of the sample means is fairly normally distributed. In addition, the more samples one takes, the more the graphed results should take the shape of a normal distribution.

The central limit theorem is often used in conjunction with the law of large numbers, which states that the average of the sample means will come closer to equaling the population mean as the sample size grows. This concept can be extremely useful in accurately predicting the characteristics of very large populations.

In general, as the sample size from the population increases, its mean gathers more closely around the population mean with a decrease in variance. Thus, as the sample size approaches infinity, the sample means approximate the normal distribution with a mean, µ, and a variance, 𝜎2/ 𝑛 . As shown above, the skewed distribution of the population does not affect the distribution of the sample means as the sample size increases. Therefore, the central limit theorem indicates that if the sample size is sufficiently large, the means of samples obtained using a random sampling with replacement are distributed normally with the mean, µ, and the variance, 𝜎2/𝑛 , regardless of the population distribution.

---

What Is a Normal Distribution?

Normal distribution, also known as the Gaussian distribution, is a probability distribution that is symmetric about the mean, showing that data near the mean are more frequent in occurrence than data far from the mean. The normal distribution appears as a "bell curve" when graphed.

In a normal distribution, mean (average), median (midpoint), and mode (most frequent observation) are equal. These values represent the peak or highest point. The distribution then falls symmetrically around the mean, the width of which is defined by the standard deviation.

The normal distribution is one type of symmetrical distribution. Symmetrical distributions occur when a dividing line produces two mirror images. Not all symmetrical distributions are normal since some data could appear as two humps or a series of hills in addition to the bell curve that indicates a normal distribution. (While the normal distribution is symmetrical, not all symmetrical distributions are normal. For example, the Student’s t, Cauchy, and logistic distributions are symmetric.)

For all normal distributions, 68.2% of the observations will appear within plus or minus one standard deviation of the mean; 95.4% will fall within +/- two standard deviations; and 99.7% within +/- three standard deviations.

This fact is sometimes called the "empirical rule," a heuristic that describes where most of the data in a normal distribution will appear. Data falling outside three standard deviations ("3-sigma") would signify rare occurrences.

 

Asymptotic Nature: The tails of the normal distribution curve approach, but never touch, the horizontal axis. This implies that all possible values of the variable, no matter how extreme, have a non-zero probability of occurring.

Skewness measures the degree of symmetry of a distribution. The normal distribution is symmetric and has a skewness of zero. If the distribution of a data set instead has a skewness less than zero, or negative skewness (left-skewness), then the left tail of the distribution is longer than the right tail; positive skewness (right-skewness) implies that the right tail of the distribution is longer than the left.

Kurtosis - Kurtosis measures the thickness of the tail ends of a distribution to the tails of a distribution. The normal distribution has a kurtosis equal to 3.0. Distributions with larger kurtosis greater than 3.0 exhibit tail data exceeding the tails of the normal distribution (e.g., five or more standard deviations from the mean). 

This excess kurtosis is known in statistics as leptokurtic, but is more colloquially known as "fat tails." The occurrence of fat tails in financial markets describes what is known as tail risk. Distributions with low kurtosis less than 3.0 (platykurtic) exhibit tails that are generally less extreme ("skinnier") than the tails of the normal distribution.

Although normal distribution is a statistical concept, its applications in finance can be limited because financial phenomena—such as expected stock-market returns—do not fall neatly within a normal distribution. Prices tend to follow more of a log-normal distribution, right-skewed and with fatter tails.

Nature and Normal Distributions

The normal distribution is technically known as the Gaussian distribution, however, it took on the terminology "normal" following scientific publications in the 19th century showing that many natural phenomena appeared to "deviate normally" from the mean. This idea of "normal variability" was made popular as the "normal curve" by the naturalist Sir Francis Galton in his 1889 work, Natural Inheritance.

Many naturally occurring phenomena appear to be normally distributed. For example, the average height of a human is roughly 175 cm (5' 9"), counting both males and females.

Also for example, if we randomly sampled 100 individuals, we would expect to see a normal distribution frequency curve for many continuous variables, such as IQ, height, weight, and blood pressure.

 

As with any probability distribution, the normal distribution describes how the values of a variable are distributed. It is the most important probability distribution in statistics because it accurately describes the distribution of values for many natural phenomena. Characteristics that are the sum of many independent processes frequently follow normal distributions. 

Another way of looking at it from reddit:  Things in nature don't "conform" to the normal distribution. The normal distribution "emerges" from the addition of a lot of small constituent events when you observe their sum. From the CLT. It happens precisely because things are random. If they are numerous enough, independent enough, and sufficiently closely distributed, you'll observe that their sum looks like coming from a normal distribution, or close to one.

 

---

 

Today, the normal distribution is ubiquitous in statistical practice for several reasons:

• It serves as a reference distribution against which other distributions are compared.
• The mathematical tractability makes it ideal for building more complex statistical models.
• It underpins many machine learning algorithms and artificial intelligence systems.
• It provides a foundation for robust statistical inference even when data slightly violates normality assumptions.

The extraordinary staying power of the normal distribution over centuries testifies to its fundamental nature in describing variation across countless domains.

 

The enduring legacy of the normal distribution isn’t that it applies perfectly to all situations—it doesn’t—but rather that it provides an elegant, mathematically tractable baseline from which to understand variation. As we continue to collect and analyze ever-larger and more complex datasets, the humble bell curve will undoubtedly remain a crucial reference point in our statistical toolkit.

--

And perhaps for a later day: (sort of setting it up as a future challenge to make sense of some of the following!)

Edwin Thompson Jaynes put it very beautifully that the max entropy distribution is “uniquely determined as the one which is maximally noncommittal with regard to missing information, in that it agrees with what is known, but expresses maximum uncertainty with respect to all other matters”. Therefore, this is the most principled choice. Here is a list of probability distributions and their corresponding maximum entropy constraints, taken from Wikipedia.

Alright, but what normal distribution has anything to do with these? It turns out that normal distribution is the distribution that maximizes information entropy under the constraint of fixed mean m and standard deviation s2  of a random variable X . So, if we know the mean and standard deviation of some data, out of all possible probability distributions, Gaussian is the one that maximizes information entropy, or, equivalently, it is the one that satisfies the least of our assumptions/biases. This principle may be viewed as expressing epistemic modesty or maximal ignorance because it makes the least strong claim on a distribution.

Above last bit from here.

32. Rivers of the world

Water is the only common substance that can exist naturally as a gas, liquid, or solid at the relatively small range of temperatures and pressures found on the Earth’s surface.

It is believed that there are over 150,000 rivers in the world, and over 226 are over 1,000 kms in length. 

Longest rivers

Following are the world's 10 longest rivers. Spread across the globe. 

1. Nile (4,132 miles, 6,650 kms)
2. Amazon (4,000 miles, 6,400 kms) - carries more water than any river

3. Yangtze - 3,915miles. Longest river to flow entirely within in one country

4. Mississippi - Missouri - Red Rock 3,710 miles (31 US states, and 2 Canadian states)

5. Yenisey-Angara-Selenga 3,442 miles (Mongolia, Southern Russia northward to Arctic Ocean)

6. Huang He - yellow river 3,395 miles

7. Ob-Irtysh 3,362 miles. China, northern russia, arctic

8. Parana 3,032 miles

9. Congo 2,900 miles

10. Amur-Argun 2,761 miles (Mongolia, Russia, China to Sea of Okhotsk)

Further, here's a map of 30 largest rivers and their drainage basins

Further, here's a look at population along some of the key river basins:

The scientists estimated that the total volume of water in Earth’s rivers on average from 1980 to 2009 was 2,246 cubic kilometers (539 cubic miles). That’s equivalent to half of Lake Michigan’s water and about 0.006 percent of all fresh water, which itself is 2.5 percent of the global volume. Despite their small proportion of all the planet’s water, rivers have been vital to humans since the earliest civilizations.

The map following shows the volume of water stored by hydrologic region. The researchers estimated that the Amazon basin (darkest blue) contains about 38 percent of the world’s river water, the most of any hydrologic region evaluated. The same basin also discharges the most water to the ocean (second map): 6,789 cubic kilometers (1,629 cubic miles) per year. That’s 18 percent of the global discharge to the ocean, which averaged 37,411 cubic kilometers (8,975 cubic miles) per year from 1980 to 2009.

In all, the Earth’s water content is about 1.39 billion cubic kilometers (331 million cubic miles), with the bulk of it, about 96.5%, being in the global oceans. As for the rest, approximately 1.7% is stored in the polar icecaps, glaciers, and permanent snow, and another 1.7% is stored in groundwater, lakes, rivers, streams, and soil. Only a thousandth of 1% of the water on Earth exists as water vapor in the atmosphere.

Water vapor—and with it energy—is carried around the globe by weather systems.

Throughout the hydrologic cycle, there are many paths that a water molecule might follow. Water at the bottom of Lake Superior may eventually rise into the atmosphere and fall as rain in Massachusetts. Runoff from the Massachusetts rain may drain into the Atlantic Ocean and circulate northeastward toward Iceland, destined to become part of a floe of sea ice, or, after evaporation to the atmosphere and precipitation as snow, part of a glacier.

Water molecules can take an immense variety of routes and branching trails that lead them again and again through the three phases of ice, liquid water, and water vapor. For instance, the water molecules that once fell 100 years ago as rain on your great- grandparents’ farmhouse in Iowa might now be falling as snow on your driveway in California. 

(One has to think of a human life, a drop of water in the larger consciousness ocean/system. One could appear anywhere on the Earth, in any form. And keep circulating, until the planet lasts. One has to believe and have faith that whatever power put everything in motion follows the same fabric of core individual balance - it could be anything, anywhere, and it would be all, all completely accessible. All the rest of human business, sentimental toppling of people is just horizontal randomness. The divine magic is unlike anything human or even other scales/intermediates powers or anything with relativity concerns. It is all here, wherever one is. )

It could be a fisherman in a Congo river rapid, fishing for living, living below 'poverty line', and yet be completely content, whole, full of good sense and good cheer, a joy to himself and anyone around. It could be a newborn baby in one of the richest households in the world, it could be a young woman behind a burka figuring out her place in the modern world trying to find her best path forward in a world she sees on screen and one she lives in, and yet because the holiest resides inside, she be a joy full of good sense and good cheer to herself and anyone around her. Or it could be young man in a modern city finding his path and purpose in the changing materials of the world, making sense of history and the times to come and the role of a human being. Each one of us is made from that same core. Pretty much accessible in our being, that power. In the dailiness of our lives, and everyday preoccuaptions, the practical and the general relativity of the social world, one forgets, it dims. And perhaps it is not really a lot. It is just this little belief that the basic principle of equation, equity, balance is itself the only truth, that there is no point of consciousness where that power is not there, or is not available all. It made the world as a creative pursuit. And the truth and goodness ensures that it could breathe as anyone, any random point and just by being the best possible or doing the best it can with available materials, it realises the higher self inside. Just being is needed. Breathing, and trying to do the best. Quite simple. But one forgets. One forgets.)

...

Nile

257 million people reside in the Nile basin. So Nile basin hosts 20% of African population

Eleven countries share the river: Burundi, the Democratic Republic of the Congo, Egypt, Eritrea, Ethiopia, Kenya, Rwanda, the Sudan, South Sudan, the United Republic of Tanzania and Uganda.

Lake Victoria with the surface area of  66,700 square kilometres is the world’s second largest freshwater lake after Lake Superior in North America.

Following are the Nile basin countries:

A comparison with other river systems:

River discharge is the amount of water that passes a given point in a river over a specific period, essentially the river's flow rate. (typically measured in cubic meters per second (m³/s).)