Foresight Institute

While at our BCI & WBE for Safe AI Workshop, Anders Sandberg discussed 'Naysayers are Your Unpaid Focus Group'.

Here, Sandberg discusses strategies for advancing the WBE community by engaging critics who can provide valuable insights to refine the community's approach. Firstly, he identifies several common arguments against WBE, including ethical concerns, technological limitations, and a lack of clear definitions for success. To address these, he proposes developing clearer metrics and benchmarks, exploring technological solutions, and organizing workshops to discuss the ethics and consequences of future WBE. He concludes that treating naysayers as an unpaid focus group can strengthen the community's strategies and make meaningful progress.

Watch the full recording here:
https://foresight.org/summary/anders-sandberg-naysayers-are-your-unpaid-focus-group-whole-brain-emulation-workshop-2024/

Our Intelligent Cooperation seminar featured Aslan Satary Dizaji's (AutocurriculaLab), 'Evolution of Communication Under Libertarian and Utilitarian Governing Systems.'

Dizaji showcased three open-source projects built upon the AI-Economist, a multi-agent reinforcement learning framework that Salesforce recently developed for mechanism design.

In the first project, the AI-Economist is extended to investigate the development of equality, prosperity, and morality under inclusive/extractive and utilitarian/libertarian political structures and institutions.

The second project expands on the AI-Economist by simulating how governing bodies and environmental resources affect a region's long-term economic development.

The third study expands on the AI-Economist by measuring how governing systems affect a society's linguistic alignment by including communication and education. Finally, Dizaji gave a brief overview of current and upcoming projects.

Despite these advancements, Dizaji believes that realistic social simulation is still far from accurately predicting changes.

https://foresight.org/summary/aslan-satary-dizaji-evolution-of-communication-under-libertarian-and-utilitarian-governing-systems/

Recently at Worldcon, Beatrice Erkers, our Director of Existential Hope, participated in the panel 'Solarpunk: Building Better Tomorrows,' alongside Future Fiction (Publisher), Rob Cameron (Brooklyn Speculative Fiction Writers), and A. M. Dellamonica (Canadian science fiction writer).

Focusing on practical politics and the experimentation of alternative technologies, Solarpunk combined ecologically sustainable solutions with art and crafts aesthetics and emerging technologies such as renewable energies, 3D printing, and AI. The panel discussed how the works of Solarpunk addressed the challenges posed to humanity, whether social, economic, or environmental, and envisioned a better future.

A special thank you to the host, Phoebe Wagner (Lycoming College), for making this event a success!

📸 Photos by Norbert Fiks.

Learn more about Existential Hope: https://www.existentialhope.com/

There is only one month left to apply to
our AI Safety Grants program. We're accepting applications in Neuroscience, Security & Cryptography, and Multipolar Approaches to AI Safety.

Application Deadline: September 30
Learn more: https://foresight.org/ai-safety/

Kristian Rönn's (Normative) presentation, 'The Path to Intelligent Cooperation' at Vision Weekend Europe, centered on his new book, "The Darwinian Trap," and its connection to AI governance and existential risks. He highlighted how natural selection, rather than AI or other contemporary threats, was the primary driver of these risks, leading to competitive pressures that pushed societies toward resource exploitation and weapon development. To counteract this, Rönn proposed the idea of 'reputational markets' to foster responsible behavior and promote global cooperation.

Full recording here:
https://foresight.org/summary/kristian-ronn-the-path-to-intelligent-cooperation-vision-weekend-europe-2024/

Attend Vision Weekend USA, December 6-8, in the Bay Area: website at foresight.org/vw2024us.

Dmitri Mitya Chklovskii (Simons Foundation)
Chklovskii discusses 'The Smart Neuron: A Biologically-Inspired Foundational Building Block of Neural Networks', a normative neuroscience approach to simulate large-scale whole-brain connectomes and understand neuronal function. He argues that current models of biological neurons are either too simplistic or overly detailed for WBE.

Chklovskii's group, Normative in Neuroscience, aims to generate a computational primitive that can be placed into the nodes of the connectome, enabling whole-brain simulation. He explains this approach as deriving a solution to an optimization problem as an algorithm that meets the constraints of biological hardware. The team then maps this algorithm onto a biological neuron, capturing essential physiological properties, such as membrane potential voltage dynamics and synaptic plasticity.

Watch more here:
https://foresight.org/summary/mitya-chklovskii-the-smart-neuron-wbe-workshop-2024/

Evan Miyazono (Atlas Computing) presented on 'Formally Scalable AI Oversight Through Specifications' as part of our Intelligent Cooperation program.

In this talk, Miyazono discusses how Atlas Computing is using this architecture to develop narrow AI systems with quantitative safety guarantees. Before starting Atlas Computing, Evan Miyazono created and led a metascience team within Protocol Labs, as well as venture studio focused on building better tools for human coordination. He holds a PhD in Applied Physics from Caltech and a BS and MS from Stanford.

Watch the full recording here:
https://foresight.org/summary/evan-miyazono-formally-scalable-ai-oversight-through-specifications/

Hendrik Dietz's talk, 'The Path to Bionanotech Progress', at Vision Weekend Europe, highlighted advancements and challenges in the fields. He began by discussing ATP synthase, a molecular machine essential for cell ATP production, illustrating its complex and efficient design as inspiration for bionanotech. Dietz then covered his research in DNA nanotechnology, showcasing successful creations such as nanoscale electric motors. He also emphasized breakthroughs in protein design driven by tools like AlphaFold, which predict protein structures from sequences. He noted the significant commercial investments in the field from companies like Cryro Therapeutics and Alto Labs. Addressing challenges, Dietz stressed the need for improved experimental validation and better integration of academic and market-driven approaches. He acknowledged limitations in silico verification due to data constraints and the complexity of understanding AI models, advocating for enhanced experimental methods and increased funding to advance the field.

Watch the recording here:
https://foresight.org/summary/hendrik-dietz-the-path-to-bionanotech-progress-vision-weekend-europe-2024/

Attend Vision Weekend USA, Dec 6-8, in the Bay Area: https://foresight.org/vw2024us/

Applications are open for Catalyze’s new incubation program for technical AI safety organizations. This program will bring together founders, advisors, and funders to advance the AI safety field.

Both non-profit and for-profit companies are eligible. Deadline to apply: September 3rd.

Apply now:
https://www.catalyze-impact.org/apply

Nikolina Lauc’s talk, 'The Path to Reliable Aging Clocks', at Vision Weekend Europe, explored the development and utility of biological age measurements. She discussed the limitations of early aging metrics, like telomere length, which reflects single-cell aging, and highlighted more comprehensive tools like the Horvath and Glycan Age clocks. These clocks provide insights into systemic aging and predict mortality and disease risk, though they often don't correlate with each other and vary in effectiveness across individuals. Lauc stressed that while aging clocks can indicate disease risks and respond to interventions, a single comprehensive biological age measure is insufficient. Personalized approaches and understanding individual differences in aging are essential for effective health management and disease prevention.

Watch the recording here: https://foresight.org/summary/nikolina-lauc-the-path-to-reliable-aging-clocks-vision-weekend-europe-2024/.

Make sure you attend Vision Weekend USA by securing your tickets here:
https://foresight.org/vw2024us/

Maria Avramidou (University of Oxford) outlined 'Alignment and Value Drift in Whole Brain Emulation', at the Foresight Neurotech, BCI, and WBE for Safe AI Workshop.

Avramidou argues that successful brain emulations should have the same values as biological humans, eliminating the need to specify initial values. Using evolutionary theory, she introduces the concepts of natural selection and replicators to understand how values and structures change over time, leading to questions about which entities are subject to natural selection, who benefits long-term, and who adapts. Avramidou suggests that an entity's fitness determines long-term benefits from selection and that behavior adaptation and value drift depend on the environment, which can be shaped for positive outcomes. She urges consideration of incentives emulations might employ, such as self-copying, self-sacrifice, and cooperation with humans, and how these can benefit humans as secondary beneficiaries.

Recording here:
https://foresight.org/summary/maria-avramidou-value-drift-in-wbe-whole-brain-emulation-workshop-2024/

Jeremy Barton presented on 'Foundational Nanomechanical Systems' at Vision Weekend Europe. His talk highlighted his new venture in molecular nanotechnology, focusing on developing foundational nanomechanical systems through open-source research and interdisciplinary collaboration. Barton shared his vision for overcoming the challenges of secretive corporate study and advancing the field through broad public involvement.

Watch the full recording here:
https://foresight.org/summary/jeremy-barton-foundational-nanomechanical-systems-vision-weekend-europe-2024/

Continuing this conversation, Jeremy Barton will chair our upcoming Nanomachines Workshop. Dates: October 4-6, 2024

Apply to Attend:
https://foresight.org/2024-foresight-molecular-manufacturing-architectures-workshop/

Catalin Mitelut (NYU) presented at our Neurotech, BCI, and WBE for Safe AI Workshop on 'Scale-dependent WBE: An Experimental Driven Research Program.'

Mitelut focuses on functional neural data recording and its applications in WBE. He aims to record rodent electrophysiology and behavior to build a neural emulator using language models, generating predictions of both behavior and neural states. Mitelut considers whether modeling neural dynamics alone could achieve WBE, bypassing extensive mechanistic neuroscience research. Particularly intrigued by the complexity of spontaneous neural dynamics and their role in the neuroscience of agency, he plans to conduct extensive electrophysiology recordings of rodents in open-field environments. By capturing these intricate dynamics, Mitelut hopes to deepen our understanding of WBE and replicate the success of language models.

Watch the full recording here:
https://foresight.org/summary/catalin-mitelut-scale-dependent-wbe-an-experimental-driven-research-program-wbe-workshop-2024/

The Survival and Flourishing Fund have launched a call for prototyping and R&D of Flexible Hardware Enabled Governance (FlexHEGs).

This is a fantastic chance for innovators working at the intersection of technology and governance to push the boundaries of what's possible.

Deadline: September 15th, EOD

https://survivalandflourishing.fund/sff-2024-flexhegs-application

Register to learn from Angela Thornton
(Certified Member of the Market Research Society) as she presents 'Our Futures in Mind Uploading: Public Perceptions and Narratives' on Thursday, September 5.

Apply to Attend:
https://lu.ma/pwwdlspa

Juan Benet (Protocol Labs) spoke at our Neurotech, BCI, and WBE for Safe AI Workshop on 'Required Innovations in Neurotech for Safe AI'.

Benet emphasizes the significance of grasping computer complexity and the promise of spiking neural network models in driving AI breakthroughs. He forecasts that transformative BCIs will emerge within the next three decades, with median AI models already rivaling human performance in randomly selected tasks.

Stressing the obstacles in constructing experimental frameworks for brain imaging, modeling, and simulation, he highlights the necessity for a field-wide R&D approach that enables rapid data sharing and iterative testing at scale. Drawing parallels to successful accelerated R&D programs such as the Apollo missions and Manhattan Project, Benet anticipates the coming 10-20 years to be a captivating period for AI advancements.

Watch the full recording here:
https://foresight.org/summary/juan-benet-accelerating-whole-brain-emulations/

We’re hosting Vision Weekend Puerto Rico in picturesque Old San Juan next year! Come for an immersive experience, which includes conferences, unconferences, mentorship hours, curated one-on-one sessions, demos, satellite events, and more.

Secure your spot now with an early bird ticket:
https://foresight.org/vw2025pr/

Allison Duettmann, CEO of the Foresight Institute, recently opened our workshop on brain-computer interfaces (BCI) and whole-brain emulation (WBE) to advance safe AI development. The event brought together around 60 researchers, scientists, entrepreneurs, funders, and activists to explore the potential of these technologies. Attendees evaluated their impact, risks, benefits, timelines, costs, and discussed new R&D mechanisms to advance these projects.

Watch the recording here:
https://foresight.org/summary/allison-duettmann-on-foresights-neurotech-bci-and-wbe-for-safe-ai-workshop-2024/

Please submit to our AI Safety Grant's neurotech category with your projects.
https://foresight.org/ai-safety/

We are excited to announce the speakers of our upcoming Functional Nanomachine Design Workshops. The workshop will take place on October 5 – 6, 2024 in San Francisco.

Our workshop chair, Jeremy Barton, is a leading expert in foundational nanomechanical systems.

Brenda Rubenstein is an Associate Professor of Chemistry and Physics (Brown University).

Christian Wagner is an ERC Group Leader (Forschungeszentrum Jülich).

Siyuan (Stella) Wang is a Foresight Fellow and Postdoctoral Fellow (Wyss Institute at Harvard University).

Don’t miss the chance to gain insights from these individuals and delve into the latest advancements in nanomachine design and nanomechanical systems.

Please apply to attend here:
https://foresight.org/2024-foresight-molecular-manufacturing-architectures-workshop/

The AGI: Cryptography, Security, and Multipolar Scenarios Workshop Report is now available.

This report offers a detailed overview of the workshop’s discussions on AI threat models, cryptographic tools, and safety strategies. Held at The Institute, Salesforce Tower, San Francisco, the event brought together sixty experts to address key issues at the intersection of AI, cryptography, and security.

Access the report here:
https://foresight.org/technologies/computation-intelligent-cooperation/intelligent-workshops-reports/

AI Safety Grantee Catalin Mitelut (Netholabs) has launched an innovative project titled 'Lo-fi Approaches for Uploading: Developing a Turing Test for Cloning Biological Organisms Using High-Precision Multi-Modal Behavior Modeling (HMBM)' in the Neuro Approaches category.

Summary: Netholabs would like to hire a graduate student/technician to analyze continuous multi-week behavior recordings from freely moving gerbils published as this dataset (Mitelut et al.et al 2023, biorxiv). Netholabs will seek to adapt inverse reinforcement learning methods to multi-agent states and build behavior models to test the Turing model for human cloning I previously proposed (Mitelut et al. et al. 2023, arxiv; see below for full description). Netholabs will additionally curate and release this dataset for public use and collaborate with the wider machine-learning community to investigate lo-fi cloning as high-precision multi-modal behavior modeling (HMBM).

For more details visit, https://www.netholabs.com/

Ramsey Brown (Mission Control AI), in his talk 'What if it Works?' At Vision Weekend, Europe examined the profound implications of reaching the technological singularity, a point where technological advancements radically transform human civilization. He employed metaphors from black hole physics to describe this shift. Brown explored the complexities and unpredictability of forecasting the future, emphasizing that technologies would reshape the socio-technical landscape like mass distorting space-time.

He noted that the singularity might arrive in phases—first gradually, then suddenly. He highlighted the importance of observing changes in scale, friction, the movement of ideas and resources, and shifts in time perception.

Brown underscored the need for core values such as truth and beauty to guide us through these changes and encouraged using these metaphors to better prepare for and understand the future.

Watch his talk here:
https://foresight.org/summary/ramsay-brown-what-if-it-works-vision-weekend-europe-2024/

Our Latest AI Safety Grantee, Leonardo Christov-Moore (Institute for Advanced Consciousness Studies), is working on his project, “Homeostasis and Extinction in Multiagent Reinforcement Learning,” which will explore innovative ways to ensure stability and safety in complex multiagent systems.

Apply for the AI Safety Grant before our next round closes September 30: https://foresight.org/ai-safety/

At Vision Weekend Europe, Chris Eliasmith (University of Waterloo) presented 'How to Build a Brain,' He explored the development of the SPaWN brain model, its capabilities, and its implications for understanding brain function and advancing technology. His presentation covered the model's evolution, its performance on various tasks, and its potential impact on fields such as AI, neuromorphics, and neuroprosthetics.

Full Recording here:
https://foresight.org/summary/chris-eliasmith-how-to-build-a-brain-vision-weekend-europe-2024/

At Vision Weekend Europe, Ben Cu***ng, the Communications Director at the Future of Life Institute (FLI), discussed FLI's mission and recent AI-related initiatives. Founded ten years ago, FLI focused on steering transformative technology to benefit humanity, evolving from a community of global experts to a policy, outreach, and grant-making organization.

Ben emphasized FLI's updated position on AI, which acknowledged AI's potential to solve significant problems while recognizing the risks associated with the unchecked development of artificial superintelligence. FLI advocated for building institutions to govern AI safely and developing AI to address specific, existing challenges.

He also discussed the importance of effective communication strategies to make these ideas resonate with the public, highlighting collaborations like the World Building Contest with the Foresight Institute. FLI's initiatives included identifying successful AI projects, such as Kate C.'s work with Amini in Africa and promoting global cooperation in AI governance.

Ben concluded by mentioning FLI's ongoing projects, including a superintelligence illustration contest and a $4 million grant scheme addressing power concentration due to advanced AI.

Watch the full recording here: https://foresight.org/summary/ben-cumming-ai-for-the-sustainable-development-goals-vision-weekend-europe-2024/

Don't miss the opportunity to engage with these visionaries shaping the future of space governance at our upcoming Workshop, 20-21 September.

We look forward to having Mehak Sarang (Open Lunar Affiliate), Victor Luo (NASA), Justin Walsh (Chapman University), and Philip Linden (Planet Labs).

Apply now and secure your spot: https://foresight.org/foresight-openlunar-space-workshop/

We are thrilled to announce the Molecular Systems Engineering Platform with Foresight co-Founder Eric Drexler.

Date: October 4, 2024
Time: 3:00 PM – 8:00 PM
Location: The Institute, Salesforce Tower, San Francisco

Apply to attend: https://foresight.org/2024-foresight-molecular-manufacturing-architectures-workshop/

In his presentation, George Hosu critiques the scientific process for favoring established paradigms and media-driven popularity over solid evidence, using the theory of electron tunneling in G protein-coupled receptors. He highlights exercise as the most effective longevity intervention but notes the lack of a comprehensive biological model explaining its efficacy or limits. Hosu compares the evolution of scientific paradigms, like those of the universe and grafting practices, to argue that novel ideas should be valued and explored despite their immediate practical utility. He also discusses his research on reversing cognitive decline through increased brain perfusion and vascularization, suggesting that enhancing blood flow to the brain can potentially reverse aging effects using simple, accessible methods.

Full recording here:
https://foresight.org/summary/george-hosu-solving-aging-by-breaking-out-of-ontologies/

Chandler Smith, from MATS, has been awarded a Foresight AI Safety Grant for his groundbreaking project on Multi-Agent Safety. Co-funded by Open Philanthropy, this research will be conducted at the prestigious Torr Vision Group under the guidance of Professor Philip Torr.

Don’t miss your chance to apply for the AI Safety Grant. The deadline is September 30th: https://foresight.org/ai-safety/

Engage with leading experts as we address innovative solutions at the upcoming Space Futures & Governance workshop.

Speakers include Guiliana Rotola (Outer Space Institute), Kevin Hubbard (Honeybee Robotics), Thomas Sebastian (MIT Lincoln Laboratory), and Chris Johnson (Secure World Foundation)

Apply today as spots are filling fast:
https://foresight.org/foresight-openlunar-space-workshop/