Build an Image Prompt Browser Extension with Natural Language Coding

    This guide walks through building a Manifest V3 browser extension that turns any image on a webpage into a ready-to-use English prompt for image search and text-to-image generation. It explains how to define requirements and scope, design a three-layer architecture, describe the spec in natural language for an AI coding tool, configure an API key, and implement the core pieces: image detection, background model calls, the result panel, and a settings page. It also covers iterative development, a testing checklist, common pitfalls with fixes, and ideas for future improvements.

    1. Background: Let Images “Speak”

    When doing design work, organizing assets, or preparing inputs for text-to-image models, we often need to write an accurate prompt for a single image. Describing a picture word by word is time-consuming and depends heavily on your own writing ability. Scenarios like image search, content archiving, and collecting inspiration all need the same thing: turning visual information into structured text.

    Multimodal large models can understand both images and text, so they can generate a natural-language description from an image and solve exactly this problem. But if you have to open a chat page, upload an image, and type an instruction every time, the workflow becomes long and disconnected from the act of browsing a webpage.

    So I set a clear goal: build a browser extension that lets you click any image on any webpage to see its prompt and copy it with one click.

    Building such an extension used to require setting up a project, writing a manifest file, handling cross-origin requests, and designing a settings page—a real barrier for anyone unfamiliar with frontend work. Today, with AI coding tools that understand natural-language requirements, you only need to describe the need in plain language, and the tool generates the project skeleton and core code. Load the extension locally, configure the model service address and key, and you are ready to use it.

    2. Solution Design: Think First, Then Build

    2.1 Define the Goal and Boundaries

    Before letting a tool write code, define the goal clearly:

    • Use case: When browsing images on any webpage, get that image’s prompt with one click;
    • Core features: Detect the page image → call a multimodal model to generate a description → display the result → one-click copy;
    • Input and output: The input is an image plus a prompt template; the output is an English prompt usable for search or further creation;
    • Boundaries: Do not automatically process every image—only respond to user clicks; do not collect usage data; keep the key stored locally only.

    2.2 Overall Architecture

    The extension has three layers:

    Layer Responsibility Key Capabilities
    Content script layer Detect images in the page, mount buttons and result panel DOM manipulation, clipboard writing
    Extension background layer Fetch image data, call the model API, handle errors uniformly Cross-origin requests, storage access
    Settings page Configure service address, API key, and model name Local persistence

    There are two reasons for putting network requests in the background rather than firing them directly from the page:

    1. On authorized domains, the extension background can bypass the page’s same-origin policy, making cross-origin calls more stable;
    2. Centralizing the model-call logic in the background keeps the content script focused only on UI interaction, which makes responsibilities clearer and errors easier to locate.

    3. Describing the Requirement in Natural Language

    Writing the requirement description extends prompt engineering into software development. A good description should include: role definition, a feature list, interaction details, technical constraints, and acceptance criteria. For example:

    1. Help me build a browser extension (Manifest V3) called "Image Prompt Assistant" with these requirements:
    2. 1. After the page loads, add a "Generate Prompt" button to the top-right corner of any image
    3. whose width and height are both no smaller than 100 pixels. Show it only on hover so it
    4. does not block content;
    5. 2. On click, send the image to a multimodal large model and have it generate an English prompt;
    6. 3. Pop up a result panel near the image showing the prompt, with one-click copy and close;
    7. 4. Provide a settings page for the API endpoint, API key, and model name, saved locally;
    8. 5. When the key is missing, image reading fails, or the API errors out, show a clear message;
    9. 6. Organize the code into modules and add comments to key logic.

    Three tips for describing requirements:

    • Give testable conditions: “width and height both no smaller than 100 pixels” is far clearer than “filter out small icons,” so the generated code will not pick an arbitrary threshold.
    • Write down acceptance criteria: Stating expectations such as “clicking the copy button must write to the clipboard” up front gives you a basis for later testing.
    • Control the granularity: Describe only one relatively complete feature at a time. With too many features at once, the tool tends to lose track and leave logic gaps.

    4. Configuring the Key: Connecting the Model Service

    The extension itself contains no model capability; it needs an external large-model service accessed through an API key. Most cloud platforms offer large-model services, where you can create an API key in the console and view the API address and model name. Taking Baidu AI Cloud as an example, you can create a key on the API key management page of its Qianfan large-model platform.

    Once you have the configuration, fill in three fields on the settings page:

    • API key: The key used for authentication;
    • API address: The API endpoint provided by the model service;
    • Model name: The name of the multimodal model being called.

    Save, and you are ready to go. Most platforms support the common conversational API format, so the endpoint, message structure, and parsing logic are largely portable across platforms. When switching providers, you usually only need to change the address and model name.

    Security note: An API key is equivalent to an account credential. Never write it directly into source code, and never expose it in screenshots or shared material. If it leaks, disable it in the console immediately and create a new one.

    5. Core Implementation

    5.1 The Manifest File: Declaring Capabilities and Entry Points

    A browser extension describes its capabilities and entry points through a manifest file. Key points:

    1. {
    2. "manifest_version": 3,
    3. "name": "Image Prompt Assistant",
    4. "version": "1.0.0",
    5. "permissions": ["storage", "clipboardWrite"],
    6. "host_permissions": ["https://your-api-domain/*"],
    7. "background": { "service_worker": "background.js" },
    8. "content_scripts": [
    9. {
    10. "matches": ["<all_urls>"],
    11. "js": ["content.js"],
    12. "css": ["content.css"],
    13. "run_at": "document_idle"
    14. }
    15. ],
    16. "action": { "default_popup": "popup.html" },
    17. "options_page": "options.html"
    18. }
    • permissions: Declares the required permissions. storage is used to read and write configuration, and clipboardWrite is used to write to the clipboard;
    • host_permissions: Declares the target addresses allowed for cross-origin access. Make sure to enter the real model service domain, or the browser will block the request;
    • content_scripts: Specifies the scripts and styles to inject into pages; document_idle means inject after the document has finished loading;
    • options_page: Points to the path of the settings page.

    5.2 Image Detection and Button Mounting

    A page may contain many images. Binding a click event to each image individually will slow the page down. A better approach is event delegation: insert a button for each image, but have all buttons share the same click handler.

    1. // Run after the document finishes loading
    2. if (document.readyState === 'loading') {
    3. document.addEventListener('DOMContentLoaded', init);
    4. } else {
    5. init();
    6. }
    7. function init() {
    8. // Event delegation: handle all button clicks in one place
    9. document.addEventListener('click', (e) => {
    10. if (e.target.closest('.pw-btn')) {
    11. e.preventDefault();
    12. e.stopPropagation();
    13. const img = e.target.closest('.pw-wrap').querySelector('img');
    14. showResult(img);
    15. }
    16. }, true);
    17. }
    18. function decorateImages() {
    19. document.querySelectorAll('img').forEach((img) => {
    20. // Key checks: fully loaded and has actual dimensions
    21. if (!img.complete || !img.naturalWidth) return;
    22. if (img.closest('.pw-wrap')) return; // Avoid duplicate insertion
    23. const w = img.getBoundingClientRect().width;
    24. const h = img.getBoundingClientRect().height;
    25. if (w < 100 || h < 100) return; // Filter out icons, avatars, and other small images
    26. const wrap = document.createElement('span');
    27. wrap.className = 'pw-wrap';
    28. img.parentNode.insertBefore(wrap, img);
    29. wrap.appendChild(img); // Keep the original image in its place on the page
    30. const btn = document.createElement('button');
    31. btn.className = 'pw-btn';
    32. btn.textContent = 'Prompt';
    33. btn.title = 'Generate image prompt';
    34. wrap.appendChild(btn);
    35. });
    36. }

    The stopPropagation() call in the handler prevents the click from bubbling up to the page’s own logic, avoiding unintended actions such as navigation. .pw-wrap uses relative positioning and the button uses absolute positioning; the wrapper uses display: inline-block to keep the original layout largely unchanged.

    Run decorateImages() once after the page finishes loading, and set up a throttled timer to incrementally process new images as the page changes:

    1. decorateImages();
    2. let timer = null;
    3. const observer = new MutationObserver(() => {
    4. clearTimeout(timer);
    5. timer = setTimeout(decorateImages, 500); // Throttle to avoid frequent scanning
    6. });
    7. observer.observe(document.body, { childList: true, subtree: true });

    5.3 Fetching the Image and Calling the Model in the Background

    Network requests are done in the background script. The content script sends the image URL to the background, which fetches the image, calls the API, and returns the result:

    1. // ===== background.js =====
    2. // Message routing
    3. chrome.runtime.onMessage.addListener((msg, _sender, sendResponse) => {
    4. if (msg.type === 'GEN_PROMPT') {
    5. handleRequest(msg.payload)
    6. .then(sendResponse)
    7. .catch((err) => sendResponse({ ok: false, error: err.message }));
    8. return true; // Keep the message channel open for the async response
    9. }
    10. });
    11. async function handleRequest({ imageUrl }) {
    12. // 1. Read local configuration
    13. const cfg = await chrome.storage.local.get(['baseUrl', 'apiKey', 'model']);
    14. if (!cfg.apiKey || !cfg.baseUrl) {
    15. throw new Error('Please fill in the API address and API key on the settings page first');
    16. }
    17. // 2. Fetch the image and convert it to a Base64 Data URL
    18. const res = await fetch(imageUrl);
    19. if (!res.ok) throw new Error('Failed to read the image; please check that it is accessible');
    20. const blob = await res.blob();
    21. const dataUrl = await new Promise((resolve, reject) => {
    22. const reader = new FileReader();
    23. reader.onload = () => resolve(reader.result);
    24. reader.onerror = reject;
    25. reader.readAsDataURL(blob);
    26. });
    27. // 3. Build a request body compatible with the OpenAI Chat Completions style
    28. const body = {
    29. model: cfg.model,
    30. temperature: 0.2,
    31. messages: [
    32. { role: 'system', content: SYSTEM_PROMPT },
    33. {
    34. role: 'user',
    35. content: [
    36. {
    37. type: 'image_url',
    38. image_url: { url: dataUrl, detail: 'low' },
    39. },
    40. ],
    41. },
    42. ],
    43. };
    44. // 4. Call the API with timeout control
    45. const controller = new AbortController();
    46. const timer = setTimeout(() => controller.abort(), 30000);
    47. let resp;
    48. try {
    49. resp = await fetch(cfg.baseUrl, {
    50. method: 'POST',
    51. headers: {
    52. 'Content-Type': 'application/json',
    53. Authorization: `Bearer ${cfg.apiKey}`,
    54. },
    55. body: JSON.stringify(body),
    56. signal: controller.signal,
    57. });
    58. } catch (e) {
    59. if (e.name === 'AbortError') throw new Error('Request timed out; please try again later');
    60. throw new Error('Network error; please check your connection');
    61. } finally {
    62. clearTimeout(timer);
    63. }
    64. // 5. Parse the result
    65. if (!resp.ok) {
    66. if (resp.status === 401) throw new Error('Invalid API key; please check your settings');
    67. if (resp.status === 429) throw new Error('Too many requests; please try again later');
    68. throw new Error(`API returned an error (${resp.status})`);
    69. }
    70. const data = await resp.json();
    71. const text = data.choices?.[0]?.message?.content;
    72. if (!text) throw new Error('Could not parse the model response');
    73. return { ok: true, prompt: text.trim() };
    74. }
    75. // System prompt: the key to determining the output style
    76. const SYSTEM_PROMPT =
    77. 'You are an expert image tagger. Describe the image in one English prompt ' +
    78. 'suitable for image search and text-to-image generation. Follow this order: ' +
    79. 'main subject; appearance details such as color and material; action or state; ' +
    80. 'background and setting; art style (photo, illustration, 3D, anime, etc.); ' +
    81. 'lighting and camera angle. Output one line only, without explanations.';

    A few notes:

    • Request format: The code above uses a format compatible with OpenAI Chat Completions. If your platform uses a different format, you only need to adjust how the body is built and how the result is parsed; the overall flow stays the same;
    • Base64 transfer: Converting the image to a Base64 Data URL in the background and passing it along with the image URL avoids the restrictions of reading cross-origin images directly from the page;
    • Timeout and status codes: AbortController controls the timeout; targeted messages for common status codes like 401 (authentication failure) and 429 (too many requests) keep users from facing a raw error string with no clue what to do.

    5.4 Displaying the Result and One-Click Copy

    The result panel is inserted directly into the page and supports copying and closing:

    1. // ===== content.js (continued) =====
    2. async function showResult(img) {
    3. let panel = document.querySelector('.pw-panel');
    4. if (panel) panel.remove();
    5. panel = document.createElement('div');
    6. panel.className = 'pw-panel';
    7. panel.innerHTML =
    8. '<div class="pw-head"><span>Prompt</span>' +
    9. '<button class="pw-copy">Copy</button>' +
    10. '<button class="pw-close">×</button></div>' +
    11. '<textarea class="pw-text" readonly placeholder="Generating…"></textarea>';
    12. img.closest('.pw-wrap').appendChild(panel);
    13. panel.querySelector('.pw-close').onclick = () => panel.remove();
    14. panel.querySelector('.pw-copy').onclick = async () => {
    15. const text = panel.querySelector('textarea').value;
    16. if (!text) return;
    17. try {
    18. await navigator.clipboard.writeText(text);
    19. const btn = panel.querySelector('.pw-copy');
    20. btn.textContent = 'Copied';
    21. setTimeout(() => (btn.textContent = 'Copy'), 1500);
    22. } catch {
    23. panel.querySelector('textarea').select();
    24. }
    25. };
    26. const reply = await chrome.runtime.sendMessage({
    27. type: 'GEN_PROMPT',
    28. payload: { imageUrl: img.src },
    29. });
    30. const textarea = panel.querySelector('textarea');
    31. if (reply?.ok) {
    32. textarea.value = reply.prompt;
    33. } else {
    34. textarea.value = reply?.error || 'Unknown error';
    35. textarea.classList.add('pw-error');
    36. }
    37. }

    The panel uses absolute positioning. If the readonly attribute on the textarea prevents text selection in certain browser engines, falling back to letting the user select and copy the text manually works as a safety net when copying fails.

    5.5 The Settings Page

    The settings page uses the simplest possible three input fields. The point is not visual polish, but validation and guidance:

    1. <!-- options.html (excerpt) -->
    2. <form id="cfg">
    3. <label>API Key <input id="apiKey" type="password" placeholder="Paste your API key" /></label>
    4. <label>API Address <input id="baseUrl" placeholder="https://your-service-address/v1/chat/completions" /></label>
    5. <label>Model Name <input id="model" placeholder="Name of a multimodal model that supports image understanding" /></label>
    6. <button type="submit">Save</button>
    7. <span id="tip"></span>
    8. </form>
    9. <script src="options.js"></script>
    1. // ===== options.js =====
    2. const fields = ['apiKey', 'baseUrl', 'model'];
    3. // Populate saved configuration when the settings page opens
    4. chrome.storage.local.get(fields).then((cfg) => {
    5. fields.forEach((k) => {
    6. document.getElementById(k).value = cfg[k] || '';
    7. });
    8. });
    9. document.getElementById('cfg').addEventListener('submit', async (e) => {
    10. e.preventDefault();
    11. const cfg = {};
    12. fields.forEach((k) => (cfg[k] = document.getElementById(k).value.trim()));
    13. if (!cfg.apiKey || !cfg.baseUrl || !cfg.model) {
    14. document.getElementById('tip').textContent = 'All three fields are required';
    15. return;
    16. }
    17. try {
    18. new URL(cfg.baseUrl); // Validate the address format
    19. } catch {
    20. document.getElementById('tip').textContent = 'Invalid API address format';
    21. return;
    22. }
    23. await chrome.storage.local.set(cfg);
    24. document.getElementById('tip').textContent = 'Saved';
    25. });

    6. Iterative Development: Getting the Tool Closer to the Requirement

    The strength of AI coding tools is that they can quickly produce runnable code, but the first version is often just a starting point. A recommended iteration order:

    1. Get the main flow working: image detection, button mounting, API calls, result display;
    2. Refine the experience: loading states, copy feedback, error messages;
    3. Handle edge cases: cross-origin images, very large images, lazy loading, settings validation;
    4. Optimize performance: request throttling, result caching, preventing duplicate submissions.

    After each round of iteration, open the “Extensions” page in the browser’s developer tools and click “Reload” to apply your changes. When you hit a problem, feed the specific error message or unexpected behavior back to the tool—for example, “the image position shifts after the button is inserted” or “a security error occurs when reading a cross-origin image.” A concrete description of the symptom is far more effective than “optimize it a bit more.”

    7. Testing and Common Pitfalls

    7.1 Test Checklist

    • Normal images: Can common webpage images generate prompts correctly;
    • Cross-origin images: Can images from a CDN be read correctly;
    • Small images: Are icons, avatars, and similar elements filtered out correctly;
    • Error scenarios: Are there clear messages when the key is missing, the key is wrong, the network is down, or the API times out;
    • Large images: Does generating a prompt for high-resolution images get noticeably slow;
    • UI interaction: Is the copy button usable, and does the panel close properly.

    7.2 Common Problems and Fixes

    Problem Cause Fix
    Cross-origin request blocked The API domain is not declared, or the request runs in the page context Declare the domain in host_permissions and keep all requests in the background script
    Content script and background communication times out The async callback does not return a response in time Return true in the listener and call sendResponse after the async work completes
    Result panel is covered by page styles The page’s global CSS affects the panel Give the panel a high enough z-index and raise selector specificity; use Shadow DOM to isolate styles when necessary
    Images are not detected Only the DOM was scanned, missing CSS background images or canvas content Expand the detection scope as needed, or clearly state which image types are supported in the requirement
    Long requests and repeated charges The same image is processed repeatedly Add result caching and request throttling, and disable the button while a request is in progress

    8. Directions for Further Improvement

    • Configurable templates: Offer multiple prompt templates in the settings page (search-oriented, image-generation-oriented, e-commerce description), switchable with one click;
    • Multilingual output: Switch between Chinese and English prompts depending on the use case;
    • Context reference: Combine the page title and surrounding paragraph text of the image’s location to make the description fit the context better;
    • Cost control: Downscale large images proportionally before uploading, and reuse results for identical images to reduce unnecessary calls;
    • Capability chaining: Send the generated result straight into a text-to-image tool to form a closed loop of “read image → describe → create again.”

    9. Conclusion

    Going from the idea of “click an image to generate a prompt,” to using an AI coding tool to produce a working extension, to filling in the key, completing the configuration, and running it for real, the whole process taught me that natural language has become a new kind of “programming language.” Whether you can quickly build a handy little tool no longer depends only on how much syntax you know, but on two things:

    • Whether you can describe the requirement clearly and specifically enough;
    • Whether you understand the principles behind the code and can judge whether a solution is sound and its boundaries are safe.

    The browser extension form is lightweight and close to the entry point, making it an ideal vehicle for packaging large-model capabilities. Following the process in this article—whether extracting prompts, translating selected text, or summarizing webpages—you can quickly build a tool of your own.