Some Limitations of ChatGPT

University of Texas president Jay Hartzell declared this to be "The Year of AI." So, the assignment for my "Mental Spa Day" at work was to be introduced to ChatGPT. The first task was to compare Google with ChatGPT.  "Choose a topic to search in both Google and ChatGPT: How did the results differ? What were you looking for and which one gave you the information that you needed?" 

The two tools are different, of course, though Google does now claim to be supported by an AI Assistant. 

I asked if Gerard Kuiper ever met Beatrice Tinsely, if Otto Struve ever met Gerard Kuiper, and if Otto Struve ever met Beatrice Tinsley. ChatGPT delivered approximately the same reply in all three cases. In that, it completely missed the fact that Struve and Kuiper worked together at the University of Chicago before Kuiper took over the McDonald Observatory from Struve.

I then asked ChatGPT: Tell me about the pay warrants of the Texian Navy, a subject for which I did independent research. My work was presented as an article in The Numismatist and as a lecture at an ANA Convention (virtual via Zoom) 4 August 2020. That talk is available on YouTube here https://www.youtube.com/watch?v=EWGzivjlOUI

ChatGPT's answer was at once cursory and inventive. Contrary to its claims, the warrants did not specify the period of service or the vessel on which the sailor served. The warrants were issued solely on the authority of Commodore Edwin Ward Moore. They were repudiated by Pres. Sam Houston. ChatGPT's offering is echoic of a college student writing to fill space when they cannot answer the question.

ChatGPT replied:

[quote] The pay warrants of the Texian Navy were a form of financial compensation issued to sailors and officers serving in the navy of the Republic of Texas during the 19th century. The Texian Navy, also known as the Texas Navy, played a crucial role in the fight for Texan independence from Mexico and in protecting the fledgling republic's coastline.

As with any military or naval force, it was essential to ensure that sailors and officers were compensated for their service. The pay warrants served as a form of promissory note, guaranteeing payment for their work and dedication to the cause of Texan independence.

The pay warrants were typically issued by the government of the Republic of Texas, often by the Navy Department or other relevant authorities responsible for managing naval affairs. These warrants were written documents specifying the amount of money owed to the individual sailor or officer, as well as any relevant details such as the period of service and the vessel on which they served.

In many cases, pay warrants could be redeemed for cash or used to purchase goods and services, depending on the policies and resources available to the Republic of Texas at the time. However, it's worth noting that the financial situation of the Republic of Texas was often precarious, and there were instances where pay warrants could not be immediately honored due to budgetary constraints or other factors.

Despite these challenges, the Texian Navy played a crucial role in safeguarding Texan interests and maintaining maritime security during a tumultuous period in the region's history. The issuance of pay warrants was a tangible acknowledgment of the sacrifices made by sailors and officers who served in the navy, ensuring that they received the compensation they were owed for their service to the Republic of Texas. [close]

PREVIOUSLY ON NECESSARY FACTS

Misconduct in Science and Research

Why Evidence is Not Enough

Springer Closes Wall of Shame

Slow Down and Think

Book Review: Blink by Malcolm Gladwell

Intuition is underappreciated. We like reams of repeatable data and validated statistics. As important as those are, they are perhaps less than half of the cognitive processing that we bring to problem solving. Of course, intuition is developed over time by experience that is informed by reason and tested by measurement. However, for the expert, whether a museum curator attributing an archaic statue or Marine Corps general repelling an invasion, intuition brings success. 

We all use it every day, especially when reading each other’s faces. Gladwell calls that “mind reading” because your face reveals your feelings—even if you try to hide them. It is “mind reading” also because making the face of an emotion triggers the same processes in the mind that external events would cause.

Blink: the Power of Thinking without Thinking
by Malcolm Gladwell,
Little, Brown and Co., 2005

Intuition also fails when it is under-informed. The classic “Pepsi Challenge” of the 1970s and 80s supposedly demonstrated that in a blind taste test, most people prefer the taste of Pepsi over Coke. And they did. But we do not drink beverages in single sips with blindfolds on. More to the point, the very average consumers had no training in tasting. Gladwell introduces us to the world of professional tasters. Through long training, they develop vocabularies and sets of measures that are integrated into their percepts. They become deeply knowledgeable about food. For them, Oreos have ninety attributes of appearance, flavor, and texture. On a scale of 1 to 15, how slippery is your mayonnaise?

But experts can be wrong if their expertise is derived from the wrong measures. Professional audiences were unenthused with The Mary Tyler Moore Show and All in the Family.  Gladwell advocates for a musician named Kenna.  (You can find him on YouTube, of course.) Kenna can’t get air time. But people who know music want his. He burst into the nightclub scene, shot his own videos, and was pursued by producers. But his music does not fit the radio genres. I listened to half a dozen on YouTube, all very good and all over the map musically.

Intuition is not omnipotent. It will not overcome physical barriers of time and space. That is what leads to the continuing string of tragedies where the police over-react and kill an unarmed person who is not threatening them. Their intuition is destroyed by the high-speed chase, the words and actions of other officers. Alone, forcing themselves to stop, they perceive the small slices that say “this man is not dangerous.” In those cases of wrongful harm, the police more closely resemble the functional autistic who is more interested in a light switch than a face. They have shut down parts of their brain thus denying intuition the opportunity to inform.

Malcolm Gladwell is a very popular writer. I only found out about his works recently while re-watching episodes of NUMB3RS. His name and his book, Tipping Point, are dropped in the Season 1 story, “Sniper Zero.” He is clear, concise, conversational, and insightful. He is a synthesizer, drawing together different stories to create a new narrative.

That brings a caveat. The presentation is not rigorous. Cook County Hospital in Chicago treats hundreds to thousands of people each day, most of them indigent and therefore without good medical support. A common problem is the man complaining of a heart attack. It must be taken seriously, but often is not an infarction. Sorting them out takes resources and erring on the side of caution requires even more bed space (and expense). Dr. Brendan Reilly began creating a database of case histories to serve a guideline in diagnosis. Reilly based his work on a previous effort by Dr. Lee Goldman who developed a mathematical model for cardiac symptoms. But Reilly met resistance, of course, from the ER staff cardiologists who did not want to surrender their expertise to an algorithm. Says Gladwell, "The algorithm doesn't feel right." But is that not the point of the book? Why would their intuition be less authoritative? This is likely just a simple misstatement. But I take it as a signal that Gladwell is offering us an array of interesting facts united by a thematic hypothesis. 

On the upside, Gladwell takes a mechanistic approach to solving the problems of social discrimination. He tells of how women finally made it into the major symphony orchestras when blind auditions became standard practice. From another source entirely I learned that India uses this to preclude geographic, ethnic, and religious bigotry from interfering with university placement. Examinations are anonymized. This week, I will again be a judge in the Austin Energy Regional Science Festival. I think that the suggestion has merit but I am not sure how to implement it in our case.

PREVIOUSLY ON NECESSARY FACTS

Slow Down and Think

The Remarkable Story of Risk  

Science versus Common Sense  

George Boole’s Laws of Thought 

Politics and the Inverse Square Law

The farther you are from something, the less you know about. In physics, this applies to the force of gravity, to electro-static attraction or repulsion, to the intensity of sound.  More generally, it applies to social and political distances.  Political decisions made at city hall, the county seat, the state capital, and Washington D.C. must suffer by the square of the distance from a lack of information.

From DigitalArtForm.com

On the other hand, when you make economic choices for yourself, your family, or household, you are most directly informed about the facts and the consequences.  The further into the future you plan, the less you know and the greater your risks.

Political plans to save the planet and the future generations that inhabit it must suffer from degradations of knowledge along every dimension of true costs, actual benefits, and both primary and secondary consequences intended and unintended.  Indeed, it is quite likely that many, if not most, future generations will not even be on this planet.

The world’s oldest continuing companies (see Wikipedia here) are all narrowly focused on mundane pursuits such as beer, wine, candy, jewelry, and hospitality.  It is hard enough to make beer successfully for 200 years.  The oldest companies in the United States (Wikipedia here) reflect the economic paradigm of capitalism: pencils, chemicals, tools, and newspapers. And we still have old breweries and continuing farms.

http://www.classroomhearing.org/images/6db_drop.jpg

 Governments are less successful.  The UK and USA are wonderful exceptions. England last executed a king in 1649.  The Glorious Revolution of 1688 defined the status of crown and parliament.  The Constitution of the United States was ratified a hundred years later.  Although Iceland does claim the longest continuing parliament, the Althing, that institution only existed within a context of subordination to Norway, Sweden, or Denmark, as Iceland was not politically independent from 1262 to 1944. Generally, governments come and go with alarming regularity exactly because they are so poor at both predicting the future and responding to new circumstances.

SUNY Canton Engineering
and Construction Management
Interactive here.

Perhaps the best judgment came from Prof. Newt Gingrich when he was Speaker of the House.  Addressing a Republican Party fund-raiser, he said of the government, “These people are not evil. They just have the wrong information system.”  In other words, they are informed by power, not market.  And the physics of power is not encouraging. Though not all analogies between electrical circuits and political action are valid, generally, any resistance can draw more power until the system overloads.   

PREVIOUSLY ON NECESSARY FACTS 

The Science of Liberty

The Remarkable Story of Risk

Entrepreneurship

Space is the Place: Come to the High Frontier

Visualizing Complex Data

Edward R. Tufte's classic works - Envisioning Information, Visual Explanation, and The Visual Display of Quantitative Information - are milestones in applied epistemology.  Drawings convey meanings across verbal barriers.  But at some level, they must be learned.  A Martian watching a baseball game could probably figure out about half of it in the first two innings. But the other "half" is actually the 90% that brings joy and sorrow to fans.  The classic No Parking sign seems universal, but could mean almost anything.  The Journal of Irreproducible Results published a textured, grainy, and striated image that could save you money by being included as an example in many kinds of research.  (Unfortunately, some scientists do re-use their data, and other people's data, in just such a manner.  See Misconduct in Scientific Research on this blog.)  Humor and fraud help define the limits of the acceptable interpretations of images.  Yet, like a linguistic gold standard, one picture remains worth 1000 words, no matter how many new words we invent.

Many philosophical Objectivists easily assert that in the hierarchy of knowledge metaphysics is the base upon which epistemology rests. However, Leonard Peikoff wrote that the two studies are dependent on each other.

With the Yin-Yang of Metaphysics-Epistemology at the core
the orbiting body could be ethics (morality)
and the field that it generates would be politics.

Computers store information and we connect them in many convenient topologies. Knowledge could be mapped the same way. 

Is the connection geology-geography-geophysics-physics-chemistry a ring or a tree or a mesh?

A Historical Map of Philosophers by Simon Raper at the "Drunks and Lampposts" blog. Raper used a mathematical visualization tool called gephi (see the website here). 

The Continental Tradition is green. Purple dots are British Empiricists 

(including those not actually from the UK such as William James).  

He derived his data from Wikipedia articles citing the influence of one philosopher's work from and then to others, as easily, Plato and Aristotle, Leibniz and Descartes (teal).

The Klein Bottle is to three dimensions what the Moebius strip is to two: it folds into itself, forming a continuous space (or surface).  Here, three Klein Bottles are interconnected. Could this map the way that planetary geology suggests the changes in weather and climate that alter ecological and economic possibilities, causing political changes in social organizations and institutions?

Although he claimed to distrust philosophy, Richard Feynman was not alone among the empirical scientists who suggested that what we perceive as "particles" or "statistical waves" are only the fields which are themselves manifestations of something "deeper" or "beyond." 

In business, firms merge and divest, departments are reorganized, investors take or sell off large interests. In the multi-party politics of Europe, alliances over issues come and go. Consequences that have not been foreseen might not be mappable until after the fact.  

Previously on Necessary Facts
Knowledge Maps
The Genius of Design
Objectified
Art and Copy
Start the Presses: Typeface

Ground Truth

Geographers who work with remote sensing data call "ground truth" the final empircal test.  You never know for sure what the instruments are telling you until and unless you go there with your own natural-born senses. 

Over lunch, a couple of months ago, one of my former geography professors told me that despite two years on this field project, one of the graduate assistants became lost when faced with a detour.  Always focused on the vehicle's GPS, the driver had no sense of geographic context.  This is not unsual.

Last month, my wife had a similar experience.  Returning to her apartment from the airport, she told the taxicab driver which turn to make.  Focused on the GPS, the driver insisted on the direct route. Faced with orange barrels, they had to go four blocks around and the difference was the driver's lost tip.  The tip was included; the driver just burned it up. 

This happened to me twice when I first came to Austin.  Eyes nailed on the GPS, drivers from two different companies drove in circles looking for addresses that I could see from the back seat.  They ignored my help - eventually, my pleas.  In the first case, after my meeting, I walked back to the motel, a direct backstreet distance of a quarter-mile that cost me $10 in a taxicab via the freeways, offramps, and access roads.  After the second incident, I decided that I will never take a taxicab in Austin again.  I would rather spend the time waiting for buses, reading a book, than to spend the money on drivers who do not know their city.

I drove a cab three times in the 70s and 80s.  The first time, I was new in town. I depended on the map, of course, but as Alfred Korzybski told us, the map is not the territory.  I made notes.  My third time, though I knew the town much better, I still started out with a fresh map in booklet form, which I annotated, drawing in new subdivisions as I discovered them.  I do not blame a driver for not knowing every street, but I do hold any person morally responsible to use their own senses.

In remote sensing, we learn six ways to create "yellow." 

• A direct radiation of 5600 angstroms.
• Reflection: all other wavelengths absorbed; yellow reflected.
• Full Red and Full Blue with No Green in an RGB display
• Full Yellow and No Cyan and No Magenta in a CMYK display
• Passing White light through a Yellow filter
• Refraction of White light at an appropriate angle

It is a prinicple of objectivism (and empiricism) that the senses cannot be fooled.  When you see an "optical illusion" the trick is in the mind, not the eye.  When you see yellow, yellow is what you see.  However, as the rationalists warn us, we do not necessarily know how the yellow was created.  For that, you need "ground truth."

ALSO ON NECESSARY FACTS
In Suspect Terrain
The Map that Changed the World
Mapping it Out: An Atlas of Contemporary Cartographics
The Philosophical Breakfast Club