Flying Shear Thinking for Online Cutting and No-Pause Switching: Engineering Practices

Introduction: Bringing the Flying Shear from the Factory to Production

On a steel strip rolling line, the strip moves continuously at several meters per second, sometimes more than ten. To cut it into sheets of fixed length, the most intuitive method is to stop the strip, cut it with a shear, and then restart. However, the steel is produced as a continuous flow. Every stop imposes a high cost on line capacity, energy consumption, and labor hours. This is why the “flying shear” exists: the object being cut keeps moving, and the blade keeps moving as well. The cutting mechanism stays synchronized with the strip, keeping the cut point approximately stationary relative to the strip, and then cuts accurately at the preset length position.

In software systems, the “steel strip” is a continuously flowing data stream, and the “shear” is every change action, such as windowing, log rotation, schema upgrade, or version switching. The traditional approach is often “stop-and-cut”: stop writing or stop the service, make the change, and then resume. But most business systems today must remain available 7x24, with downtime windows that are becoming shorter or close to zero. “Flying shear thinking” addresses the same question: without stopping or slowing down the system, how can a cut be accurate, stable, and recoverable if something goes wrong?

Four Elements of the Flying Shear Model

A flying shear completes fixed-length cutting while the strip is moving. This is not about cutting faster; it depends on the coordination of four elements. These elements map closely to software systems.

1. Moving carrier: the object being processed continuously. On the strip line, it is the moving metal strip. In software systems, it may be a message stream, a data table, or online traffic.
2. Target cut: the desired cutting result. For strip steel, this is the fixed length. In software systems, it may be the window size, the split file size, the target schema, or the target version.
3. Phase and speed synchronization: the key to flying shear. The cutting mechanism must run at the same speed as the strip and operate at the correct time so that the cut lands at the intended position without tearing. In software systems, this corresponds to cut-point identification and progress catch-up: the old and new paths must process the same input at the switching moment. Any progress difference may cause duplicate data, data loss, or failed requests.
4. Cut quality: the measure of the cutting result. A good cut should be flat, burr-free, and consistent in size. In software systems, the corresponding metrics include whether the split boundaries are clean, whether data is duplicated or missing, the request success rate, latency, and the consistency rate between old and new results.

The flying shear model can therefore be summarized as an iron rule: the cut point must be definable, the cutting speed must be synchronizable, the cut result must be verifiable, and failure must be recoverable. The methods below are concrete applications of this rule in different scenarios.

Scenario 1: Windowing Stream Data with Out-of-Order Compensation

The most common “cut” in stream processing is slicing an unbounded data stream into time windows for aggregation, billing, or alerting. The difficulty is that when the blade drops, the data may not yet be complete.

• Event time takes priority over processing time. Window boundaries must be anchored to the event time carried by the data. If windows are cut by the time the system receives data, network latency or upstream retries may place the same business record in the wrong window.
• Use watermarks to express that data has basically arrived. A watermark is a logical point in time that advances with processing. It means that data earlier than it has basically arrived. A window should trigger only after the watermark passes its end time, just as a flying shear waits until the cut point enters the valid cutting section before cutting.
• Allow late data and route it separately. Even after a window is triggered, a small amount of late data may still arrive. A reasonable approach is to configure an allowed lateness period. Late records update the original window result and cause re-emission. Records beyond the tolerance threshold should be written to a side output stream for review or compensation, rather than being silently discarded.
• Persist first, then commit offsets. The order between persisting computation results and committing consumption offsets must be correct. A common approach is to commit the offset only after the result has been successfully written to a transactional or idempotency-keyed storage. If the operation fails, replay from the last committed offset, and let idempotent writes absorb duplicate data.
• Make every stage replayable. Use general cloud capabilities such as object storage or log services to retain raw data. If the splitting logic fails, rerun from the source instead of adding patches on top of incorrect results.

# Pseudocode: event-time window assignment and late-data routing
def assign_window(event, cfg):
    # Use event time as the only anchor; use integer milliseconds consistently to avoid timezone mixing
    idx = (event.event_time - cfg.epoch_ms) // cfg.window_ms
    watermark = current_watermark()          # Current watermark (event time)
    end_ms = cfg.epoch_ms + (idx + 1) * cfg.window_ms
    if watermark < end_ms:                   # Window is not closed yet: count normally
        return ("active", idx)
    elif watermark <= end_ms + cfg.late_grace_ms:
        return ("late_update", idx)          # Allowed lateness: update the window and re-emit
    else:
        return ("side_output", idx)          # Overdue data: route to side output for review/compensation

Scenario 2: Online Log Rotation and Rolling Collection

A log file can be viewed as a continuously “rolled” log strip. Splitting it into files by size or time is online rotation. If the cut is handled poorly, three types of problems may occur:

• Lost lines: the writer process still holds a handle to the old file. After rotation, new content is written into the renamed old file, but the collector no longer tracks it.
• Duplicated lines: during the rotation moment, both the old and new files are read, causing duplicate collection.
• Collection stalls: copying the entire rotated file and then clearing it may block writes for a long time, especially when the file is large.

The mature engineering practice is to make “cut, register, and hand over” an indivisible closed loop:

1. Atomic switch: the log writer triggers rotation by fixed size or time. The new file is created first, and then an atomic rename points the “current file” to the new file. After that, new writes only go to the new file, and the old file stops changing after a certain point in time.
2. Stable naming and state registration: the old file uses a stable name with a sequence number or timestamp and is not overwritten. The rotation event is written to a state record containing the file path, start and end time, start and end offsets, and checksum. This record is the single source of truth for the collector.
3. Offset tracking by the collector: the collector reads files in the order of the state records and persists “filename + read offset”. A file is marked complete only after it has been read to the end and confirmed to be expired. After a process restart, collection resumes from the offset, ensuring no duplicates or omissions.
4. Backpressure and rate limiting: centralized log services usually have write quotas. The collection side should use local buffering, batch compression, and rate limiting to avoid downstream congestion back-pressuring the business process.

When using cloud log services for collection and retrieval, three principles are recommended:

• identify the source and version on the collection side;
• query by time range on the retrieval side;
• archive raw logs to object storage according to the retention period.

This makes the start, end, and attribution of each cut traceable.

Scenario 3: Online Schema Change and Canary Traffic Switching

Database table schema changes and traffic switching between old and new systems are the most typical “flying shear” challenges. The table is constantly read and written, and traffic keeps flowing. The change must apply to every row of data without stopping the online “strip”.

Online Schema Change: The Five-Stage Method

Take the example of “adding a column, changing a column type, and rebuilding an index on a large table”. Executing the change on the table directly would cause long locking periods. The common online change approach in the industry is highly consistent with the flying shear principle:

1. Create a shadow table: create a new table with the target structure while the source table continues normal reads and writes. This is like installing the new blade while the production line keeps running.
2. Capture incremental changes: use a change data capture mechanism to continuously synchronize inserts, deletes, and updates from the source table to the shadow table, keeping the two “strips” moving together.
3. Backfill historical data in batches: split the data by primary key range, read, transform, and write in small batches, strictly control the speed, and avoid business peaks. If new changes occur in the source table during backfill, the order must be correct: changes captured after backfill must override the backfilled data, and old values must not overwrite new values.
4. Full verification: compare the source table and the shadow table block by block, including row counts, checksums, and sampled results of key fields. Only after all differences are cleared can the process proceed to the next step.
5. Atomic switch and observation-period cleanup: complete the switch through an atomic rename at the metadata level. The application side does not need to restart. Keep the old table for a period as a rollback fallback, and clean it up after the observation period ends.

# Pseudocode: five-stage online schema change (interfaces are generic placeholders)
def online_schema_change(db, table, ddl, cfg):
    shadow = f"{table}__new"
    # Stage 1: create a shadow table with the target structure
    db.execute(ddl.replace(":table", shadow))
    # Stage 2: start change capture and write changes to the shadow table
    stream = db.start_change_capture(table, target=shadow)
    # Stage 3: backfill historical data in batches with rate limiting
    for pk_range in db.split_primary_key_range(table, cfg.batch_rows):
        rows = db.query(f"SELECT * FROM {table} WHERE pk BETWEEN ? AND ?",
                        pk_range[0], pk_range[1])
        db.upsert_many(shadow, transform(rows))      # perform structural transformation before writing
        stream.apply_pending()                       # apply pending changes before the next batch
        throttle(cfg.max_qps)
    stream.catch_up(timeout=cfg.catch_up_timeout)    # catch up incremental changes with the source table
    # Stage 4: verify block by block; abort if inconsistent and do not proceed to switching
    for pk_range in db.split_primary_key_range(table, cfg.check_rows):
        assert db.checksum(table, pk_range) == db.checksum(shadow, pk_range)
    # Stage 5: atomic switch; keep the old table until the observation period ends
    db.execute(f"ALTER TABLE {table} RENAME TO {table}__old")
    db.execute(f"ALTER TABLE {shadow} RENAME TO {table}")
    return {"rollback_table": f"{table}__old", "observe_until": now() + cfg.observe_window}

The sample code intentionally relies only on general database capabilities such as query, write, verification, and rename, without binding to the proprietary syntax of any specific database. During implementation, pay attention to:

• use versioned migration scripts to manage the change order and ensure consistency across environments;
• record audit logs for the change process itself.

Dual-Run and Rollback During Switching

Metadata switching is only the first step. If the old and new tables are read and written by different application versions at the same time, or if the change occurs between services, a more challenging question must be answered: At the moment the blade drops, where is the traffic?

• Read path: prioritize sending all writes to the new structure. During the observation period, reads should remain compatible with the old structure. Use the new structure as the basis for comparison, and remove the old path only after the difference rate drops to zero.
• Write path: the shadow table and source table are synchronized before the switch. The truly affected part at the switching moment is whether the SQL generated by the application matches the new structure. Therefore, application upgrades should follow “compatibility first, switch later”: first release a version that is compatible with both the old and new structures, then switch the structure, and finally remove compatibility code.
• Canary traffic switching: distribute traffic by consistent hashing on stable dimensions such as user ID, region, or tenant. Requests from the same user should always land on the same version to avoid session jitter. Gradually increase the traffic ratio, and set an observation period and explicit rollback threshold for each stage. The one-click rollback switch should be rehearsed in advance, not verified only after an incident occurs.

If the relevant services are deployed on container offerings of Baidu AI Cloud, capabilities such as rolling updates, readiness probes, and graceful connection shutdown can be used to achieve instance-level seamless replacement. However, smooth instance-level replacement does not equal data-level correctness. Schema and data migration must still follow the five-stage method above.

Common Pitfalls and Launch Checklist

Six Common Pitfalls

1. Missing cut point: no clear cut point is defined. During the switch, both the old and new paths may write at the same time, or neither may write.
2. Mixed time semantics: streaming windows mix event time and processing time, or data across time zones is not converted to a unified time scale.
3. Ignoring the slow path: only the normal path is stress-tested. Large transactions, lock waits, backfill, and business peaks are not considered together.
4. Superficial verification: only the total row count is compared, without checking content distribution or boundary records.
5. No rollback path: the switching script only handles forward progress, and the rollback plan has never been rehearsed.
6. Missed cleanup: after the observation period ends, old tables, dual-write code, and shadow tasks are not cleaned up, wasting resources and leaving hidden consistency risks.

Launch Checklist

• Is the cut point defined as a concrete event, such as a version number, timestamp, offset, or business switch?
• Can the old and new paths process the same input, with only one of them responsible for output at a certain moment?
• Are streaming windows anchored to event time, with watermark, allowed lateness, and side output configured?
• Are external writes idempotent, and will failed replays produce duplicates?
• Does the online change include a shadow table, incremental capture, batch backfill, full verification, and atomic switching?
• Does data verification cover row counts, checksums, and boundary samples?
• Has the rollback plan been rehearsed, and can the switching switch take effect with one click?
• Do key metrics cover cut quality: duplicate rate, loss rate, latency, success rate, and consistency rate between old and new results?
• Are these metrics connected to cloud monitoring and alerting with tiered thresholds?
• Is raw data replayable, and does the retention period meet compliance requirements?
• After the observation period ends, is there a clear cleanup plan for the old structure and dual-write logic?

Conclusion

The value of flying shear is not in “cutting”, but in “cutting continuously while placing every cut exactly where it should be”. Online changes in software systems are the same. What determines the success or failure of a switch is not how fast the switch command is executed, but whether the cut point is clear, whether the old and new paths are synchronized, whether the result is verifiable, and whether failure is recoverable. By solidifying this approach into a process and checklist, and using the infrastructure, observability, and data retention provided by general cloud capabilities, high-frequency changes can move from “high-risk events” to “repeatable routine operations”.