Read my list of open questions/docket of musings

Read on Substack for better experience

A mega consolidated brain dump. Some are ideas I would personally want to work on and some are general musings/would be cools. Inspired by Dwarkesh, Patrick Collison, and Gwern.

Table of Contents:

1. Open research interests (AI/ML, earth systems, finance, policy)
2. Deep/hard tech (defense, robotics, infrastructure, climate)
3. Software (B2B, consumer, micro SaaS)
4. General/misc (skills, investing)

---

Open Research Interests

AI/ML in Earth + Planetary Systems

Mechanistic interpretability of ML weather models

Can we extract meaningful insights (knowledge/physics) from neural networks in the physical sciences?

• Experiment with toy/smaller transformer-based weather prediction models
• Sparse neural network ideas?
• Seems to mostly be research on transformers-based models
  • Links: Dario Amodei, Neel Nanda
  • Neel Nanda puts it: "I do not see this as a f*cked up non-interpretable black-box"
• For LLMs → cognition
• For applied physics problems → physics equations that govern it
  • General ideas of how to approach the problem

Inference + test-time compute ideas on weather prediction models

• Particularly transformer based

Approaches to optimizing and building better weather prediction models

• Human-data dual system approach: JAMES commentary
• Transformer-based: Windbourne's Weather Mesh
• Large weather models (LWM) GitHub repo
  • Ocean wave forecasting, marine weather models, aviation weather models?
  • Finetuning for data sparse regions?
• Less blurring?

Running through various scenarios/timelines of Earth as synthetic data

• Incorporating these apocalyptic situations into the models?

Synthetic data (data upscale/downscale)

Good paper on ideas:

• Reconstruction in space (spatial interpolation) — Africa weather problem
• Reconstruction in time (temporal interpolation)
• Reconstruction of scales (downscaling, superresolution)
• Reconstruction as a probabilistic problem

Learning to weather the weather

• Merlin Labs autonomous navigation — primer on how to achieve autonomous navigation // imitation learning

Data 'cleanliness'

• Cleaning up weather datasets using neural networks
• LLM for captioning remote sensing data

Better understanding ice-sea dynamics

• In a computational/modelling perspective
• Ice-sea governor

Actionable Earth Observation insights from LLMs

• Paper 1: ArXiv
• Paper 2: ArXiv with GitHub repo

Time series prediction + remote sensing

• Lightweight-transformer for remote sensing

Thought: Weather systems prediction as a boundary value problem

• What does that tell us?

Applications of visual language models in earth systems science?

IceNet incremental research

• IceNet.ai

---

General ML, Agents, Alignment Problems

Better understand the transformer circuit work by Anthropic + Olah

• 2025 research
• Anthropic Blog, Neel Nanda Video Walkthrough

Search engine embeddings via 'next-link' prediction

• Instead of next-token prediction, then using some web search API

Adapting existing regulation

• What modifications of existing regulation are needed to stably and safely integrate AI agents into society (Alan Chan)

How to govern and signpost AI agents? AI agents safety?

• While 'AI agents' is often used to refer to building web/SaaS, there's a pretty interesting field of AI agents academic/frontier research

General research topics in transfer learning/imitation learning

Parsimonious neural networks

• Material/nano science paper

HNet research

• Best way for finding the ideal tokenizer

AI policy for Hong Kong

---

Applied ML/Algo Research

Visual-language-action models for robotics

• See the deep defense tech idea below

Wine research

• Using machine learning for grapevine phenological modelling → wine futures/options trading?

Getting a better grasp of reinforcement learning

• Currently know near 0 on this topic

COLREGS and HK traffic compliant autonomous boat systems

• Collision avoidance
• Port operations

Optimising class allocation algorithms

• Dartmouth Thesis, More, Colab

Ideas in astro/planetary/space systems

• SpectraFM - finetuning models in astro
• Asteroid manipulation
• Asteroid prediction

---

Finance/Economics/Policy/International Relations

Correlation of weather patterns and commodities prices

• Commodity futures, commodity-linked stocks
  • Overlaps of weather prediction and stock market predictions
• General quantitative/financial market benefits with better weather prediction
• Inspired by Eduardo Saverin betting on oil via meteorology

HVAC, trade (plumbing etc.) and their role in private equity businesses

• New IT systems/SaaS for HVAC (sell to the PE)
• Note to self: try to find that Twitter thread
  • Software: $650 billion
  • HVAC: $500 billion in the US

The mathematics and simulations of betting markets

• Are they more often right because people have money to their word?

Should frontier AI companies stay private? (or IPO?)

• Societal/public investor pressure if there is mass ownership
• Can 'dumb money' change trajectory of AI progress?
• Similarities to the dotcom boom?

Arctic (Greenland, Svalbard) defense, trading and strategic potential

• Opened new routes with climate change

General topics in AI safety and control

• Emil will know much more

Hong Kong: policy for a tech ecosystem

• Encourage agentic
• No ghost startups
• Architect of Prosperity: Sir John Cowperthwaite and the Making of Hong Kong

Better understand how energy markets work

• e.g. Denmark and Singapore
• ML overlap?

Better understand international relations

• Extends my exposure at the HKETO

Policy: What are towns giving up to have big tech build in their state/town

• Stargate and lower tax revenue

Gov graph visualisation

• SF Gov
• Twitter thread

Fast rise to powers and leadership

• Peng Zhao of Citadel
• Damola Adamolekun of Red Lobster (ex Goldman + TPG)

Founding of Saudi Aramco

Macrohistory

Fall and rise of civilisations

---

Thoughts/Sociology/(Pseudo) Philosophy

Better understand how to most optimally help society

• With sufficient physical involvement and personal satisfaction (effective altruism linkages)

What does it mean to have taste?

• Taste of problems, taste of solutions

Cultural and aesthetic shallowness

How effective people study and learn hard concepts

Solving market-driven problems vs fundamental problems

• Market-driven problems are gaps that present themselves because of a marketplace and supply vs demand
• Fundamental problems are links of curiosity
  • Problems driven for scientific understanding
  • Problems driven by moral standards and virtue (though definition can be a little wishy-washy)
  • Problems that don't yield immediate primary economic value but can be foundations for downstream economic opportunity

One way I like to find ideas is to go to conferences/workshops (e.g. latest ICLR/ICML) and see the long/short papers and use it as a springboard for ideas of topics I am interested in. Another good way to learn is via conference workshop tutorials or equivalent for practical step-by-step guides for cool research ideas.

---

Deep/Hard Technology

Defense/Robotics/Autonomy

Mostly inspired by interest in what Palmer Luckey is doing with Anduril.

Visual-Language-Action (VLA) for Robotics

• Coordination of...
• Huggingface's OpenVLA model?
• Scout AI

Autonomous boats/sea gliders

• Modularity in design for construction
• Interesting ones: Saronic
• Ways to optimize:
  • Get sharper turns using extra wings?

Underwater drones

• Twitter thread

GPS-free autonomous navigations

Thought: How to build a Hong Kong hard tech government contracting system

Maritime autonomy

• Orca AI ($70M Series B)

Robotic pacer for the running track

• Calibrated for the track
• Integrate with your heart rate
• Object detection for other people in your track but continues the pace

---

Infrastructure

Underwater data centers

• Increase water usage
• Inspired by Highlander and this YC startup

Infrastructure projects in the developing global south

Data centers next to oil drilling platforms

• Like Crusoe
  • In Kazakhstan? Indonesia?
  • Is anyone doing this in China?
  • Compute (GPU) is location agnostic compared to cloud computing (internet)

Data centers in space?

• Starcloud

Drones to clean windows of highrise buildings

• Trains etc.

---

Earth + Climate (Taming Nature...)

Constellation of weather balloons

• Windbourne systems
• Particularly SEA/typhoon regions or low data African regions
• Hardware problem + software problem
• Balloons have greater...
• Sorcerer

Deep sea + earth exploration

• Untapped resources

Cloud seeding

• Rainmaker
• Anti-ice drones
• Anti-ice boats?
• Aeolus labs

Subsea intelligence

• Kraken Robotics

---

Software (B2B and Consumer)

Agentic social search system

• Search for finding people: academics/finance/lawyers (networking/cold-emailing)
• Holmes AI
• Vectorising

Dayrooms: app built for birthdays

• Allow you to record a quick message without awkward messages
• Good old CRUD app (probably build with Expo)

A tool to help academics find and preprocess their code better

• Start with oceanography domain and maybe like ERA5 analysis
  • Complicated to know what you want, where to get it
  • One stop place to download and preprocess that data you want
  • Like a standard for preprocessing data?
• AI driven data preprocessing prompt
• Inspired by that bio guy with biology open data

Web search

• What Exa.ai is doing with the next link prediction

Better interface to store questions + do questions

• e.g. past papers

General: bridging American SaaS into a Cantonese/Chinese version and UI

• Granola.ai

Society simulations

• What Devansh over at Decision Labs is doing

Something to do with apartment/housing/property market in Hong Kong

Evals

• Like Soren.ai

Reinforcement learning environments

Expert data source

AI engine optimisation (the prompting company)

AI-first companies

• Design: Semiotic
• Investment bank: AI first investment bank OffDeal
  • They have some nice stats on SMEs in America
• Market research: Diligence Squared

AI in PE

• Zarna AI

Physical intelligence

Trade compliance

• Trava (YC) - Use Trava
• Focused on HK/Singapore markets

---

High Value Skills

Starting an AI safety + consulting NGO for Hong Kong

• Go into companies to talk about their pain points
• Individual sessions on how to boost
  • Partner with companies??

Organising events + communities

Podcasts as a way to talk to people

• Get better at researching people
• Get better at asking questions + conversing

Starting an AI safety / solidifying the EA club at HKUST

---

Miscellaneous Topics of Interest

Explore the space economy and what opportunities there are

The Chinese tech ecosystem

• Miracle Plus, Shenzhen (tech+manufacturing) and how the companies got to where they are (Tencent, Baidu, Pinduoduo, Xiaohongshu)

Investing/market directions:

• Robotics and autonomous vehicles
• Quantum computing
• Ocean tech + robotics
• Deep sea mining
• Nuclear and uranium (particularly in growth economies)
• Satellite / space (ASTS, RKLB, LUNR)
• Defense as a whole / cybersecurity
• Global warming-caused shipping changes (Greenland + Arctic)
• Vertical supply chain of semiconductors/chip manufacturing

Bear cases of tech trends

• Bear case of TSMC, bear case of NVIDIA etc.

AI alignment tips

• Rohin Shah's FAQ

---

Programming/CS/Electronics Skills

Truly and more intuitively understand advanced topics in Machine Learning

Not just implementation, but the mathematical intuition behind the algorithms.

Useful resources:

1. Illustrated Transformer
2. Transformer for SWE
3. Distill.pub (best resource!)

Go through Neel Nanda's mech interp basics

• Barebones pre-reqs

Do some Leetcode (All of Neetcode 150)

Robotics and missile systems

Electronics: power systems, circuits and build your own board/microcontroller, sensors & actuators

• First circuits
• Ohm's Law
• Multimeter usage
• Breadboarding
• LEDs

Mechanics: CAD, materials science (briefly), 3D printing and basic mechanical engineering principles

• Shigley's Mechanical Engineering Design

Software side: vision, robotics operating system, algorithm design

Aerodynamics (more for rocketry): basics of lift, drag, thrust and weight

• Structural design
• Stress, strain and material failure
• NASA's guide to aeronautics+aerodynamics

---

Footnotes

I will probably update this consistently with new ideas.

I think if you pick a really hard research problem, try to solve it in <8 weeks, blog everything, shoot videos, ship good OSS code and repeat it, you can get hired by a big lab in ~1 year.

open research interests: ai/ml in earth + planetary systems

• mechanistic interpretability of ML weather models (more broadly physical sciences)

can we extract meaningful insights (knowledge/physics) from neural networks in the physical sciences?

• experiment with toy/smaller transformer-based weather prediction models

• sparse neutal network ideas?

• seems to mostly be research on transformers - transformers-based models

• links: Dario Amodei, Neel Nanda

• Neel Nanda puts its it: “I do not see this as a f*cked up non-interpretable black-box”

• for LLMs —> cognition.

• for applied physics problems —> physics equations that govern it

• General ideas of how to approach the problem

• inference + test-time compute ideas on weather prediction models?

• particularly transformer based

• approaches to optimising and builder better weather prediction models

• human-data dual system approach: JAMES commentary

• Transformer-based: Windbourne's Weather Mesh

• large weather models (LWM) github repo

• Ocean wave forecasting, Marine weather models, Aviation weather models?

• finetuning for data sparse regions?

• less bluring?

• running through various so-called ‘scenarios/timelines’ of the earth as synthetic data for training this models

• incorporating these apocalyptic situations into the models?

• synthetic data (data upscale/downscale) - good paper of ideas

• reconstruction in space (spatial interpolation)

• Africa weather problem

• reconstruction in time (temporal interpolation)

• reconstruction of scales (downscaling, superresolution)

• reconstruction as a probabilistic problem

• learning to weather the weather

• https://merlinlabs.com/autonomous-navigation/ —> primer on how to achieve autonomous navigation // imitation learning

• data ‘cleanliness’ - cleaning up weather data sets using neural networks

• LLM for captioning remote sensing data

• better understanding ice-sea dynamics in a computational/modelling perspective

• ice-sea governor

• actionable Earth Observation insights from LLMs

• Paper 1: https://arxiv.org/abs/2404.15500

• Paper 2: https://arxiv.org/pdf/2504.04319 (has github repo for ai agent)

• time series prediction + remote sensing

• Lightweight-transformer for remote sensing paper

• thought: weather systems prediction as a boundary value problem

• what does that tell us?

• applications visual language models in earth systems science?

• IceNet incremental research

• <https://icenet.ai/#about-us>

general ML, agents, alignment problems

• better understand the transfomer circuit work by Anthropic + Olah

• 2025 research

• Anthropic Blog, Neel Nanda Video Walkthrough

• search engine embeddings via ‘next-link’ prediction instead of next-token prediction and then using some web search API

• adapting existing regulation: What modifications of existing regulation are needed to stably and safely integrate AI agents into society (Alan Chan)

• how to govern and signpost AI agents? AI agents safety?

• while ‘AI agents’ is often used to refer to building web/saas, there’s a pretty interesting field of AI agents academic/frontier research

• general research topics in transfer learning/imitation learning

• parsimonious neural networks

• Material/Nano science paper

• hnet research

• best way for findign the ideal tokenizer a

• AI policy for hong kong

applied ML/algo research

• Visual-langauge-action models for robotics

• See the deep defense tech idea below

• Wine research

• Using machine learning for grapevine phenological modelling —> wine futures/options trading?

• getting a better grasp of reinforcement learning

• currently know near 0 on this topic

• COLREGS and HK traffic compliant autonomous boat systems

• Colision avoidance

• port operations

• optimising class allocation algorithms

• Dartmouth Thesis, More, Colab

• ideas in astro/planetary/space systems:

• SpetraFM - finetuning models in astro

• <https://www.visionofearth.org/wp-content/uploads/2019/07/Vision-of-Earth-Asteroid-Manipulation-Poster.pdf>

• asteroid prediction

finance/economics/policy/international relations

• correlation of weather patterns and commodities prices

• commodity futures, commodity-linked stocks

• overlaps of weather prediction and stock market predictions

• general quantitative/financial market benefits with better weather prediction

• inspired by Edurardo Saverin betting on oil via metereology

• HVAC, trade (plumbing etc.) and their role in private equity businesses

• New IT systems/SAAS for HVAC (sell to the PE)

• note to self: try to find that twitter thread

• software: 650 billion

• hvac: 500 billion in the US

• the mathematics and simulations of betting markets

• are they more often right because people have money to their word

• should frontier AI companies stay private? (or IPO?)

• societal/public investor pressure if there is mass ownership

• can ‘dumb money’ change trajectory of AI progress?

• similarities to the dotcom boom?

• Arctic (greenland, svalbard) defense, trading and strategic potential with the arctic ocean melting

• opened new routes with climate change

• general topics in AI safety and control

• Emil will know much more

• Hong Kong: policy for a tech ecosystem

• Encourage agentic

• No ghost startups

• Architect of Prosperity: Sir John Cowperthwaite and the Making of Hong Kong

• better understand how energy markets work (e.g Denmark and Singapore)

• ML overlap?

• better understand international relations

• extends my exposure at the HKETO

• policy: what are towns giving up to have big tech build in their state/town

• stargate and lower tax revenue

• gov graph visualisation:

• <https://sfgov.civlab.org/>

• <https://x.com/m_atoms/status/1869430358347002221>

• fast rise to powers and leadership

• Peng Zhao of Citadel

• Damola Adamolekun of Red Lobster (ex Goldman + TPG)

• Founding of saudi aramco

• Macrohistory

• Fall and rise of civilisaitons

thoughts/sociology/(pseudo) philosophy

• better understand how to most optimally help society with sufficient physical involvement and personal satisfaction (effective altruism linkages)

• what does it mean to have taste?

• taste of problems, taste of solutions

• cultural and aesthetic shallowness

• how effective people study and learn hard concepts

• solving market-driven problems vs fundamental problems

• market-driven problems are gaps that present themselves because of a marketplace and supply vs demand

• fundamental problems are inks of curiosity

• problems driven for scientific understanding

• problems driven by moral standards and virtue

• though definition can be a little wishy-washy

• problems that don’t yield immediate primary economic value but can be foundations for downstream economic opportunity

One way I like to find ideas is to go on conferences/workshops (e.g latest ICLR/ICML) and see the long/short papers and use it as a spring board for ideas of topics I am interested in. Another good way to learn are via conference workshop tutorials or equivalent for practical step by step guides for cool research ideas. deep/hard technology: Defense/robotics/autonomy: (Mostly inspired by interest of what Palmer Luckey is doing with Anduril)

• visual-Language-Action (VLA) for Robotics

• Coordination of

• Huggingface’s OpenVLA model?

• Scout AI

• autonomous boats/sea gliders

• Modularity in design for construction

• Interesting ones: Saronic

• Ways to optimise:

• Get sharper turns using extra wings?

• underwater drones

• <https://x.com/ruima/status/1998482875029815726?s=12>

• GPS-free autonomous navigations

• thought: how to build a Hong Kong hard tech government contracting system

• maritime autonomy:

• <https://www.orca-ai.io/> (70mil series B)

• robotic pacer for the running track

• calibrated for the track

• integrate with your heart rate

• object detection for other people in your track but continues the pace

Infrastructure

• underwater data centers

• Increase water usage

• Inspired by Highlander and this YC startup

• infrastructure projects in the developing global south

• data centers next to oil drilling platforms

• like crusoe,

• in Kazkahstan? Indonesia?

• Is anyone doing this in China?

• Compute (GPU) is location agnostic compared to cloud computing (internet)

• data centers in space?

• Starcloud

• drones to clean windows of highrise buildings

• trains etc,.

Earth + Climate (taming nature…)

• constellation of weather balloons:

• Windbourne systems

• particularly SEA/typhoon regions or low data African regions

• hardware problem + software problem

• balloons have greater

• Sorcerer

• deep sea + earth exploration

• Untapped resources

• Cloud seeding

• rainmaker

• Anti-ice drones

• Anti-ice boats?

• Aeolus labs

• Subsea intelligence

• https://www.krakenrobotics.com/wp-content/uploads/2025/09/Kraken-Robotics-Sept-2025-Website.pdf

• Kraken robotics

software (b2b and consumer)

• agentic social search system: Search for finding people academics/finance/lawyers (networking/cold-emailing)

• Holmes AI

• vectorising

• dayrooms: app built for birthdays and allow you to record a quick message without awkward messages

• Good old CRUD app (proabably build with Expo)

• a tool to help academics find and preprocess their code better

• start with oceanography domain and maybe like era5 analysis

• complicated to know what you want, where to get it

• one stop place to download and preprocess that data you want

• like a standard for preprocessing dat?

• ai driven data preprocessing prompt

• inspire by that bio guy with biology open data

• web search

• what exa.ai are doing with teh next link prediction

• better interface to store questions + do questions (e.g past papers)

• General: bridging American SaaS into a Cantonese/Chinese version and UI

1. granola.ai

• society simulations

• What Devansh over at Decision Labs are doing

• something to do with apartment/housing/property market in Hong Kong

• evals (like soren.ai)

• reinforcement learning environments

• expert data source

• ai engine optimisation (the prompting company)

• ai-first companies

• design: <https://www.semiotic.so/>

• investment bank: ai first investment bank <https://offdeal.io/>

• they have some nice stats on SMEs in America

• market research: <https://www.diligencesquared.com/>

• ai in pe: <https://www.zarnaai.com/>

• physical intelligence

• trade compliance

• trava (YC) - https://www.usetrava.com/

• focussed on HK/Singapore markets.

high value skills:

• starting an AI safety + consulting NGO for Hong Kong

• go into companies to talk about their pain points

• individual sessions on how to boost

• partner with companies??

• pro

• organising events + communities

• podcasts as a way to talk to people

• get better at researching people

• get better at asking questions + conversing

• starting a ai safety / solidfying the EA club at HKUST

miscellaneous topics of interest:

• explore the space economy and what opportunities there are

• the chinese tech ecosystem (Miracle Plus), Shenzhen (tech+manufacturing) and how the companies got to where they are (tencent, baidu pinduoduo, xiaohongshu)

• investing/market directions:

• robotics and autonomous vehicles

• quantum computing

• ocean tech + robotics

• deep sea mining

• nuclear and uranium (particularly in growth economies)

• satellite / space (asts, rklb, lunr)

• defense as a whole / cybersecurity

• global warning-caused shipping changes (greenland + arctic)

• vertical supply chain of semiconductors/chip manufacturing

• bear cases of tech trends (bear case of tsmc, bear case of nvidia etc.)

• ai alignment tips: <https://rohinshah.com/faq-career-advice-for-ai-alignment-researchers/>

Programming/CS/Electronics skills:

• truly and more intuitively understand advanced topics in Machine Learning - not just implementation, but the mathematical intuition behind the algorithms

(link to useful resources I wanna go through - on Notion)

1. Illustrated Transformer

1. transformer for SWE

1. distill.pub (best resource!)

• go through neel nanda’s mech interp basics:

• barebones pre-reqs

• do some Leetcode (All of Neetcode 150)

• robotics and missile systems

• Electronics: power systems, circuits and build your own board/microcontroller, sensors & actuators

• First circuits, Ohm's Law, multimeter usage, breadboarding, LEDs

• Mechanics: CAD, materials science (briefly), 3d printing and basic mechanical engineering principles

• https://www.amazon.com/Shigleys-Mechanical-Engineering-Design-McGraw-Hill/dp/0073398209

• Software side: vision, robotics operating system, algorithm design

• Aerodynamics (more for rocketry): basics of lift, drag, thrust and weight

• structural design

• stress, strain and material failure

• NASA’s guide to aeronautics+aerodyanmics

• https://www1.grc.nasa.gov/beginners-guide-to-aeronautics/learn-about-aerodynamics/

Footnotes: I will probably update this consistently with new ideas. I think if you pick a really hard research problem, try to solve it in <8 weeks, blog everything, shoot videos, ship good oss code and repeat it. you can get hired by a big lab in ~1 year. (twitter) 1 Like © 2026 Hayden So · Privacy ∙ Terms ∙ Collection notice Start your SubstackGet the app Substack is the home for great culture