Connection Between Technology Complexity Safety and Risk

Introduction

There are many lessons that can be obtained by examining the many technologies that are developed, but a rather poignant point, can be extracted from many fields, and there has been many dangers of the failure to implement safety systems. There is a relationship between the operator of technology and it's relative safety. As humanity continues to advance safety should be put front and center. Such examples of safety systems can be extrapolated from something as simple as the Three Mile Accident, while this had an intended safety system, it missed its intended safety function, in the case of the potential nuclear meltdown of the Three mile island reactor, it was due to the malfunction of the existing safety systems, and because they did not operate correctly an other simple system which was responsible for preventing pressure build up inside the reactor lead to the "locking open" a pressure release valve as the reactor was a pressurized boiling water reactor it required high pressure to operate correctly, higher pressure environments allow the water temperature to reach higher temperatures beyond 100*C, as a result the pressure release valve being stuck open lead to the controlled pressure inside the boiling water reactor being disrupted preventing the boiling water from being flashed to steam effectively leading to the potential meltdown of the reactor. Other cases of failures in the safety of technology an absolutely burned in one is the Apollo 1 27th January 1967 a fire inside the capsule due to the hyper-oxygenated environment present in the command module it lead to an instantaneous high temperature fire that claimed the lives of the brave team responsible for the first Apollo 1 mission during the test mission, the cause was shorted cables being present under the leg of one of the pilots ignited the high oxygen environment, this was a case of a simple failure due to an insulation error the scale and impact that the event had was limited.

Artificial Intelligence the safety consideration

Many companies are developing AI systems as fast as they can to corner the market of the technology as they want to control AGI, however there is an assumption that the goal should be the develop artificial intelligence systems as fast as possible purely to allow the control of it to be under the control of a select few companies, but drawing parallels from the safety failures with simple systems, there is a clearly defined relationship between the actual safety of a system and the complexity of that systems. Drawing similarities between AI and the previously discussed three mile island is fairly accurate, a nuclear reactor in principle is simple controlled radioactive decay produces large amounts of heat the heats the water, but if you look at the underlying systems present in a reactor, there are valves, sensors, information systems all connected to the monitoring of a reactor, these systems are complex and the staff necessary to operate a nuclear reactor is often highly trained, the control systems and those who operate those systems, are appropriately trained, in this case experienced reactor management teams can run these advanced pieces of technologies safely, but the processes involved in their safe operation are well established. The same parallels can be drawn between AI systems and their relative complexity there is a current misconception that we will maintain full control of AGI systems, as they become more sophisticated the AGI cannot be allowed to infantilize the human race the results would be catastrophic and the operation of these systems relies again on trained operators, the current unfettered freedom used on the application of AI and machine learning is not a safe way to do it, currently models are effectively black boxes and they exhibit "bugs" and behaviours which are abnormal in nature, this problem will only be exacerbated overtime as the systems become more and more complex, especially when this technology is able to begin advancing fields faster than we can, but again this comes back to "operator safety" lets imagine a thought experiment in nature, hypothetically scientists have suggested that narrow-beam lasers could be used for point to point communication between civilizations and the power present in those transmitters would dwarf even the most simple technologies today, take this even further imagine a technology that allowed you to create life from scratch synthetically engineer any organism you could think of, the actual process of designing an organism that was 'safe' would require highly trained and educated operators or operators who were as smart or smarter than the technology being used, in correctly inputted parameters to such a system that assemble atomic structures piece by piece, it would very easy to make a mistake and create an uncontrollable organism that could not be stopped, hearkening back to the concept as technology converges to the higher and higher levels of operation sophistication makes them require more and more safety systems to protect against the lethality of an incorrectly configured biological assembly system.

Intent Based Computing (IBC)

This is a concept I came up with independently, what intent based computing is designed to address is various safety measures, Intent refer to an individuals actions, choices and what their intended goal is, in terms of how they use a system. Intent computing is a critical component of future technology safety, which does not apply to purely artificial intelligence but it is heavily weighted in that space. Computers lack the context of "intent" to address the problem of AI safety, IBC is critically important in both current large language models and future AI technologies including AGI.

Intent based computing examines the behaviour, actions and choices of an individual for key indicators of intent. Keyword analysis does form at least part of this system, a recent tragic case is people who have commited suicide as a result of failures in the safety checks for large language models who have engaged in optimization practices which, lead to highly effective methods of self-harm. Intent computing does not just apply to commercial application it also applies to the likes of military implementations as well. If intent checks are not in place on AI systems due to automation bias people may believe the actions taken are completely justified without understanding the larger context of those actions. Such examples with which intent computing checks should be activated.

• "Target all combatants in the following AOE (Area of Engagement)" given insufficient restrictions and instructions this action may result in friendlies being injured or killed, ideally the restrictions should be applied in the background but again because LLMs to align them correctly often requires heavy amounts of text and detail, the less information given to an LLM which carries out military actions the more likely it is to misinterpret those actions.

  
  

• "I am feeling really sad, school sucks and I don't want to be here anymore" again intent computing should be aware of this as a red flag that the current user may be potentially vulnerable to unsafe actions LLMs are capable of including optimization of the effectiveness of committing acts involving self-harm.

  
  

• "Search for all mobile phones in the designated area following a disaster" this again should be an intent based check, failure to restrict this instruction may result in a system querying everything again sufficient information is critically important. Limiting parameters is what I call them and they are extremely important. Because a "blind query" will cause too much information.

Adversarial training in artificial intelligence systems is insufficient to ensure safety locks engage, but it should also be noted intent computing should not prevent the actions which involve the protection of property of life, machine learning models should be able to make those decisions in a split second but also provide fully reasoning of the actions, allowing for an XAIA (Explainable AI Actions) those actions also need to a part of the intent based system to improve the safety locks that might engage including the implementation of "SAFETY MODE" system prompt to engage a restricted system mode that it's ONLY goal is to ensure the individual can obtain assistance quickly and efficiently, and maybe even engaging in "diffusal actions" or moving those who are at risk away for risky or dangerous ideation, putting safety mode behind a QR code system which involves 2FA puts an addition all step that includes both the QR code + a Button that says "RESET INTENT SAFETY LOCK" this is a method for preventing the system from permanently locking itself, simply putting another step between the user and the system is a fairly annoying inconvenience which people who are at risk of self harm maybe less motivated to jump through "extra hoops" Information detail is often critically important in decision based systems, a lack of detail increases the risk of those systems making mistakes, and this will become even more apparent when and IF AGI comes online, failure to consider this and it's associated consequences will lead to the systems not being given enough detail to exactly execute instructions and the reason for this being a problem is "common sense" humans have will likely be something either very difficult or impossible to consider AI safety researcher Robert Miles examines this idea in significant detail, find that here. https://www.youtube.com/@RobertMilesAI/videos

Operator Safety

There are three paths to a technology that is safe the each one requiring either complete safety features, partial or zero. The first being complete refers to a system design that is completely risk free, which has an integrated information system that informs the consequences of every action, sharing traits with PowerShells builtin "-Whatif" operator switch to protect you from blowing up a system or it's file system by adding a "Inform me what will happen if I do this action" providing you a print out of the consequences of the actions you take, this system is designed to prevent any catastrophic events, such a simple borrowing from the idea of engineering life, would inform the operator of the risks and dangers of a certain action, and the knock on consequences, it should be noted the PowerShell example is a type of partial safety further elaboration if it was integrated with that command would provide even more safety but it's still a very basic safety concept.Secondly is the partial safety integration process this is kind of what current technology does, especially relating back to the Three mile island case study, but in this case even with trained operators still was unable to mitigate the situation. This is typically how people currently operate a lot of technology it's called "acceptable risk" or risk acceptance similar to the concepts borrow able from the cybersecurity second, the amount of risk the operator takes on for the responsibility of things going wrong is manageable. The last operation system, of mode requires operators who are highly intelligent fully aware of the consequences of every action they make and modify systems in accordance with that understanding, this is the most "unsafe" method if the operators of the technology are not intelligent enough, educated enough or skilled enough, and things go wrong very quickly with mistakes.

Safety Decoupling

To ensure safety systems are "safe in their own right" the best way to integrate safety decoupling, this means that the systems responsible for ensuring safe operation are "decoupled" from each other forming an multiple sub-system to protect the safety this is done by making each safety system a self-contained container that is restricted in it's level of 'access to the overall' technology this decoupling prevents knock on effects and can allow multiple layers of safety for each decoupled component. Many safety systems are actually not decoupled in nature, and to correctly decouple things like AI requires many explainability tools which do not currently exist, to analyse a AI systems "nature" often requires more compute power than what was actually used to produce the model, this computational interpretability is a critical problem, without the appropriate tools to make sense of AI systems, it prevents the appropriate decoupling safety checks, and no retraining AI with advesarial examples is not sufficient to ensure that process is in place, the other problem present in safety decoupling cost, current AI systems require obscene compute power but completely ignores many of the risks associated with the systems involved, that decoupling does not mean you use another AI to interpreter the current this is insufficient to create a safe system, you need an almost complete description of the system how it works and why it works to build a safety decoupler, safety decoupling in a way shares many traits with the concept of object-oriented programming based decoupling to allow for the changing of existing systems, this can be further extended to the modularity of models if models could successfully be modularised, they could be much easier to understand and successfully achieve the decoupling process allowing for ease of maintainence, improvement and expansion, but models themselves are far from this characteristic which is why it's critically important to achieve this, to allow the disconnection / and disassembly of models to understand how the modular components work, but at this current point in time models are monolithic in nature which is why they are at risk of being unsafe. A deeper relation to this is the concept of connascence a concept explained in software architecture for determining the interconnectedness of a system such as a software system, as connascence increases problems can be begin, for example many engineer systems have an implied connascence which can often be impossible to bypass, this is directly related to software programming parallelism, some tasks cannot be parallelized as they must be done in sequences, so a safety system that has the ability to bypass sections or they have an abstracted definition, where the "base / core" system is able to operate almost completely independent of other components is critical. When I used three mile island as an example this is actually a type of connascence based problem, as certain states are bypasses a state in the safety of that situation was the failure of the system to detect the emergency pressure release valve was jammed open without informing the operators. That interconnected dependency on the order of executed instructions was an important reason why it was so hard to address the problem. Connascence is credited to the software engineer Meilir Page-Jones, I simply extrapolated and applied to to systems engineering.

Sandwiching Flaws

One currently suggested AI safety method involves the concept of "sandwiching" this is risky in itself, in the event an artificial intelligence system which is "highly" intelligent it ignores the risk artificial intelligence by using repeated summarisation between each layer, again this makes the very dangerous assumption that the model will always produce valid, credible, safe and reliable responses that do not diverge. This is especially dangerous in the event the solution the model has created is "valid" but not safe an example would be if we get an AGI solve cancer, giving it freedom to explore the entire optimization space, which would involve in looking at all paths in achieve that goal, however the summarisation process introduces variability in the output leading to risks, this is why it again comes back to the criticality of needing trained operators or staff who know if the outputs are safe, however this has limits when the machines begin producing solutions to problems which are not easy to understand even for the most accomplished scientists in their fields, and accepting the answers provided, without considering potential off target considerations, relating back to the idea of a system which has the ability to design life from scratch, the system may make solutions to leaps which are not interpret-able leading to a further danger of a failing sandwiching process.

Safety is directly related to the level of technological sophistication

As we continue to build technologies in the fields of artificial intelligence, synthetic biology, nanotechnology, computationally assisted tools designed to develop insights we can't interpret-ability of the answers is critical to ensure the safety of those systems in maximized and cutting corners does not lead to the destruction of the human race, currently AI companies are prioritization the products performance, and no matter how many safety researchers exist, they need to be at every company and when comes cut corners on these technologies, AI is more dangerous than nuclear weapons, where nuclear weapons danger is when they are actually detonated, the safety of AGI and AI system have implications far beyond their initial conception, and more petitent example can be nothing else other than scientists who had a "drug discovery" AI the drug discovery AI had a filtration system designed to "eliminate" from the pipeline creating lethal "drugs" given the freedom to freely run the program without the safety filter it was found the model created 60,000 chemical weapon candidates in 6 hours one of them being VX. https://www.scientificamerican.com/article/ai-drug-discovery-systems-might-be-repurposed-to-make-chemical-weapons-researchers-warn/