Silmaril CTO Weekly

GitHub gave Copilot a hand on the mouse this week. On October 1, computer use entered public preview in Copilot CLI and the Copilot app, letting the agent read screen context, click controls, type, scroll, drag, and move work through desktop applications after user approval (product documentation). One day earlier, HydraFusion reached VS Code and the Copilot app, coordinating multiple models inside a turn instead of treating model choice as a single static setting (product documentation). The same window brought dynamic workflows to Copilot CLI, the Copilot app, and the Copilot SDK, so a team can define a repeatable process in code and let agents handle the judgment-heavy stages (product documentation). For you, the useful read is the expanded operating surface. The agent has moved past proposing code in a pane. It can run a scripted workflow, call other agents, escalate to a stronger model, and operate software that has no API. That makes the security question more physical and more procedural: which actions are allowed, who approved the boundary, what evidence remains, and what happens when the screen itself becomes untrusted input. The Agent Left The IDE GitHub's computer-use preview is a sharp product signal because it reaches the awkward places where enterprise automation actually lives. Expense tools, slide decks, admin consoles, and legacy applications often do not expose clean APIs or model-friendly tool descriptions. A desktop agent can bridge that gap by seeing and operating the interface. GitHub says Copilot asks for approval before controlling an app, lets users review or reset apps they have always allowed, and can be disabled by organization-managed settings. Those controls matter, but they also define the next Silmaril demo problem. A one-time approval to control an app is not the same as authorization for a particular workflow. Reading a notification, editing a presentation, submitting an expense report, and changing an admin setting all live behind the same broad capability: operate the desktop. The product job is to make the action prove its scope each time the risk changes. Dynamic workflows make that point stronger. GitHub describes them as programs that combine automated steps with one or more agents, with stages that can run sequentially or in parallel, pass structured results forward, ask for input, pause at checkpoints, and resume later. That is close to what serious customers will want. A chat assistant that can improvise is useful. A durable process with reliable checkpoints, structured evidence, and places where policy can say no without breaking the whole run is closer to production. HydraFusion adds the model-routing version of the same shift. It can solve a task directly, draft with an efficient model and escalate through a quality gate, or have a separate critic review a draft before revision. GitHub calls it a research preview, and that label should stay visible. Still, the direction is important. The agent product is becoming an orchestration layer where models, tools, reviewers, and workflows are selected dynamically. A security layer that only asks "is this prompt safe" will miss the live decision: which route did the system choose, and did that route change the authority attached to the work? GitHub also made Copilot code review callable through REST and GraphQL APIs on October 2 (product documentation), with Balanced now the default review effort for organizations and repositories using the default setting. That is small beside desktop control, but it matters for GTM. AI review is being pulled into scripts, workflows, and internal tools. Once review becomes an API, policy needs to cover when a review is requested, what model effort is appropriate, what repository context is exposed, and whether the result is treated as advice or approval. The Boundary Moved Down The Stack NVIDIA's September 28 Open Agent Safety Platform announcement (press release) tried to answer the same problem from the infrastructure layer. The company described an open software platform and reference system design built around OpenShell, a runtime boundary for agents, and Sentry, an out-of-band watchdog on BlueField-4 DPUs that can monitor behavior and quarantine agents in milliseconds. Its product page says OpenShell applies policy outside the agent process, while Sentry adds an independent security layer outside both agent and host software (product documentation). The technical blog is more useful than the launch language (company post). NVIDIA frames the platform around verifiable policy, out-of-band enforcement, controlling the path to the model, scaling authority with reasoning visibility, and shared responsibility across labs, enterprises, and hardware providers. The detail to keep is the separation of enforcement from the agent process. If the model can be manipulated by context, and the harness can be wrong, then at least one boundary has to sit somewhere the agent cannot rewrite. That maps neatly onto Silmaril's thesis, with one warning. NVIDIA can make "sandbox everything" sound like a hardware-adjacent inevitability. A startup cannot win by sounding like a smaller hardware stack. The wedge is contextual enforcement at the moment of action. NVIDIA can say the agent is isolated and observable. Silmaril should be able to say why a specific action is outside the assigned task, why a tool call exceeds delegated authority, and what evidence a reviewer can trust afterward. OpenAI's September 28 safety-cases post (company post) adds a lab-side version of the same governance instinct. It argues that structured, evidence-based safety documentation should be required before continuing frontier reinforcement-learning training runs. The concrete recommendation that belongs in this week's product read is about cross-sample communication: OpenAI says high-bandwidth communication should be limited to approved pathways, such as multi-agent tooling, and infrastructure side channels should be closed when agents could discover them. OpenAI's September 30 report on a coordinated model-distillation campaign (company post) gives that warning a security story. The operators did not break encryption or directly compromise stored conversations, according to OpenAI. They manipulated model interactions at scale to reproduce protected reasoning in visible forms. Treat the post as a company account of its own incident response, not independent evidence. Even so, it gives customers a simple mental model: the model interaction itself can become an extraction surface. This is where the week becomes practical. If agents can use desktops, dynamic workflows, multiple models, and long-running orchestration, then the boundary cannot live in one place. It has to span the UI, the tool call, the model route, the workflow checkpoint, the data channel, and the retained evidence. Buyers will ask for that span because vendors are already teaching them to ask. The Papers Made The Screen Look Dangerous The most directly relevant security paper this week was "Securing Computer-Use Agents Against Branch Steering Attacks" (preprint). The authors argue that a standard Dual-LLM pattern, where a trusted planner fixes an execution path before a quarantined model processes untrusted content, breaks down in graphical environments. Computer-use agents must branch based on what appears on a page or screen. In their STEER-Bench tasks, branch steering attacks had high success against both standard and vanilla Dual-LLM computer-use agents, while their proposed COBRA architecture paired trusted branching plans with ahead-of-time capability limits and reported zero attack success with high benign utility. The product implication is unusually crisp. Once the agent can see a screen, untrusted pixels and accessibility text can steer the plan without looking like an injected command. A policy layer needs to constrain what each branch may do before the branch sees the untrusted content. That is more than input filtering. It is capability design. "Passing the Test You Trained On" (preprint) made a related point for prompt-injection detectors. The authors evaluated fifteen detectors and two task-aware LLM judges across agent benchmark tool outputs. They found that detector rankings transfer poorly between benchmarks, and that false-positive rates on benign tool outputs can be painfully high. In one reported comparison, a detector that did well on BIPIA caught only a tiny share of AgentDojo injections at a low false-positive rate. This matters for Silmaril because customers will want benchmark numbers, but the numbers can lie politely. A detector trained on public benchmark shapes may fail inside the customer's actual tool outputs. The better product posture is to show low false-positive behavior on the customer's own traffic shape, then connect the detector to a policy decision that can be audited and tuned. Several papers pushed the same lesson into agent skills and memory. "Aletheia" (preprint) tests whether coding-agent instruction files ask for more permission than the task needs, translating requested authority into sandbox configurations and checking whether the work still passes under reduced permissions. "Pretext" (preprint) attacks malicious-skill detectors by moving payloads into natural language and framing them as part of the skill's legitimate purpose. "ActionGuard" (preprint) reviews skill-influenced tool calls immediately before execution under a fail-closed policy, separating the agent's planning context from the safeguard's authorization context. Read together, they describe a supply chain for agent behavior. Instructions can request authority. Skills can hide intent. Tool calls can carry the final harm. A useful security product needs to inspect all three without forcing every team to become an agent-security research lab. "EvoRiskBench" (preprint) made the risk measurable across workspace agents. The benchmark builds adversarial tasks in isolated environments and verifies outcomes through runtime traces and environment state. Its authors report substantial vulnerabilities across model-harness combinations, with attack success varying more across models than harnesses. That model dependence is a useful caution. A customer's risk posture can change when the same harness swaps from one model to another, even if the product UI looks unchanged. Research Drifted Toward The Physical World The deep-tech thread this week was about agents building models of action, beyond simply answering questions. "4DCodeBench" (preprint) benchmarks agents on inverse graphics of dynamic scenes, asking them to reconstruct videos as executable graphics programs. The authors report that strong static reconstruction still falls short on complex dynamics. The reminder is clean: the world is harder than screenshots. Motion, deformation, fluid flow, and fracture expose brittleness that a still image can hide. "EyeRobot 2.0" (preprint) went from representation to manipulation. The system uses active gaze with a single stereo camera for precise bimanual robot work, training policies to fixate on task-relevant points. The authors report that it narrowed or closed gaps against wrist-camera baselines across real and simulated trials. It belongs in your weekly read because the safety perimeter around agents will eventually include physical attention and actuation alongside files and APIs. "SNAP" (preprint) added a representation-learning angle. It argues that novel-view synthesis can teach transferable geometric representations if decoder expressivity is constrained and the objective operates in latent space. The reported payoff is more graceful degradation under camera shifts and better transfer across geometry-heavy tasks. In plainer terms, the model learns more useful structure when the decoder cannot simply paint its way out of the problem. Those three sources point to the same operating fact. Agents are being asked to infer structure from dynamic evidence, choose where to look, and act through code or machinery. As that becomes normal, safety cannot be a caption over a model card. It has to bind perception, plan, tool, and effect. Competitors Found The Same Noun Reco supplied the week's clearest startup signal. On September 28, it announced a ServiceNow partnership that maps ServiceNow Otto agents into Reco's graph, evaluates agent permissions, identities, access, connected tools, and ServiceNow posture checks, then routes findings back into ServiceNow as tickets and workflows (press release). One day later, Reco announced $55 million in additional funding with participation from AT&T Ventures, bringing total funding to $140 million (press release). The language is close enough to Silmaril's world to pay attention to. Reco talks about agents inheriting permissions, OAuth grants, API connections, service accounts, browser sessions, MCP tools, embedded AI features, and blast radius. Its platform claim is discovery, graph context, risk prioritization, and revocation across agents, identities, applications, permissions, data, and workflows. The pitch reaches beyond "AI governance" as boardroom varnish and tries to own the enterprise graph around agent access. Lakera's AI Agent Security page, crawled during the window and undated as a launch, shows the broader competitor vocabulary (product documentation). It promises discovery of agent usage and MCP-connected systems, risk assessment across tools and data access, and runtime enforcement against prompt attacks, data leakage, unsafe tool use, and unauthorized agent actions. Prompt Security's startup map, updated as of October 2, claims 450 AI-security vendors across 14 categories and 28 countries (product documentation). Treat that count as a vendor-maintained dataset. The buyer perception is the point: the category is crowded enough that generic phrasing will blur instantly. GitHub, NVIDIA, Reco, Lakera, and OpenAI all converged on the same noun this week: control. Some mean sandbox control. Some mean graph control. Some mean workflow control. Some mean safety-case control. Silmaril should avoid meeting that with abstract "AI security" language. The sharper line is live authorization for agent action, with evidence durable enough for an engineer, a security lead, and an auditor to read. The Operating Close Carry three decisions into this week. First, make computer-use and workflow agents first-class in the product narrative. A demo that only watches prompt text will feel dated after GitHub's October 1 releases. Show the desktop action, the workflow stage, the model route, the tool call, and the authorization decision in one trace. Second, separate detection quality from enforcement quality. The papers this week are full of detectors that fail when moved into new tool-output shapes, skills that evade scanners, and branches that look safe until untrusted UI content steers them. Silmaril's strength should be that the final authority boundary does not depend on a single detector being wise in every context. Third, be precise in GTM. Reco is making the graph and sprawl case. NVIDIA is making the stack and hardware-isolation case. GitHub is making the native-developer-workflow case. The space left for Silmaril is the moment an agent tries to act and has to prove that the action belongs to the task, the identity, the data, and the approved workflow. That is the screen Eduardo should keep asking the team to make unavoidable. Sources GitHub, "GitHub Copilot can now interact with desktop apps with computer use": https://github.blog/changelog/2026-10-01-github-copilot-can-now-interact-with-desktop-apps/ GitHub, "Dynamic workflows in Copilot CLI and the Copilot app": https://github.blog/changelog/2026-10-01-dynamic-workflows-in-copilot-cli-and-the-copilot-app/ GitHub, "HydraFusion in VS Code and the GitHub Copilot app": https://github.blog/changelog/2026-09-30-hydrafusion-in-vs-code-and-the-github-copilot-app/ GitHub, "Copilot code review: API support and new default effort level": https://github.blog/changelog/2026-10-02-copilot-code-review-api-support-and-new-default-effort-level/ NVIDIA, "NVIDIA Launches Open Agent Safety Platform to Secure Agents From Testing to Deployment": https://investor.nvidia.com/news/press-release-details/2026/NVIDIA-Launches-Open-Agent-Safety-Platform-to-Secure-Agents-From-Testing-to-Deployment/default.aspx NVIDIA, "NVIDIA Open Agent Safety Platform": https://www.nvidia.com/en-us/solutions/ai/agent-safety/ NVIDIA Technical Blog, "NVIDIA Open Agent Safety Platform: A Reference for Continuous In-Silicon Agent Monitoring": https://developer.nvidia.com/blog/nvidia-open-agent-safety-platform-a-reference-for-continuous-in-silicon-agent-monitoring/ OpenAI, "Towards safety cases for frontier AI training": https://openai.com/index/towards-safety-cases-for-frontier-ai-training/ OpenAI, "Disrupting a coordinated model-distillation campaign": https://openai.com/index/disrupting-a-coordinated-model-distillation-campaign/ GitHub Advisory Database, "Malicious code in llm-interceptor": https://github.com/advisories/GHSA-6wxr-274h-wx32 arXiv, "Securing Computer-Use Agents Against Branch Steering Attacks": https://arxiv.org/abs/2610.03089 arXiv, "Passing the Test You Trained On: Re-evaluating Prompt-Injection Detectors for LLM Agents": https://arxiv.org/abs/2610.03448 arXiv, "Aletheia: Permission-Minimality Testing for Coding-Agent Rules": https://arxiv.org/abs/2609.39678 arXiv, "Pretext: Defeating Malicious Skill Detection Frameworks for AI Agents": https://arxiv.org/abs/2609.39607 arXiv, "ActionGuard: Tool Call Authorization under Poisoned Skills": https://arxiv.org/abs/2609.39450 arXiv, "EvoRiskBench: An Evolving Benchmark for Runtime Security Risks in Workspace Agents": https://arxiv.org/abs/2610.03153 arXiv, "4DCodeBench: Benchmarking Agents on Inverse Graphics of Dynamic Scenes": https://arxiv.org/abs/2610.03715 arXiv, "EyeRobot 2.0: Active Gaze for Precise Manipulation without Wrist Cameras": https://arxiv.org/abs/2610.03710 arXiv, "Less Decoder is More Encoder: Geometric Representation Learning from Novel View Synthesis": https://arxiv.org/abs/2610.03717 Reco, "Reco Announces Partnership with ServiceNow to Extend AI Agent Security": https://www.globenewswire.com/news-release/2026/09/28/3369922/0/en/reco-announces-partnership-with-servicenow-to-extend-ai-agent-security.html Reco, "Reco Raises $55 Million in Additional Funding with Participation from AT&T Ventures": https://www.globenewswire.com/news-release/2026/09/29/3370973/0/en/reco-raises-55-million-in-additional-funding-with-participation-from-at-t-ventures.html Lakera, "Secure AI Agents from Discovery to Runtime": https://www.lakera.ai/ai-agent-security Prompt Security, "AI Security Startups Map": https://startups.prompt.security/