KittyMacro/README.md

# KittyMacro

Macros for **Anime Vanguards** and for **falling-notes events** — outside the
game, like Salmon. Nothing is injected and nothing is executed inside Roblox: it
moves the real mouse, presses real keys, and reads the screen the way you do.
From the game's side there is nothing to find, because there is nothing there —
a keyboard and a mouse doing what a keyboard and a mouse do.

Windows only. Python 3.9+. Nothing to install — it is the standard library and
`ctypes`.

## The window

```powershell
python macro.py gui          # or double-click KittyMacro.bat
```

Six pages, and a log along the bottom that tells you what the macro is doing
while Roblox is sitting in front of it:

| | |
|---|---|
| **Home** | whether this is going to work — Roblox open, build order recorded, the two points that matter calibrated — and the button that starts it |
| **Routines** | what you have recorded, which of them the macro uses, and the button that records another |
| **Calibrate** | the live cursor readout, and the five points, each showing where it is and the colour it expects to find there |
| **Piano** | the lanes of a falling-notes event, and what each one is reading right now |
| **Event** | the way into the event, when a song counts as over, and the forty runs |
| **Settings** | every number in the config file, with the line that says why you would touch it |

**Playing** at the top of the sidebar picks which macro START runs.

Every button in it sends the press the F-key sends, so the two can never
disagree about what is running: START is F7, Stop is F8, Pick waits for F6.

That matters more than it sounds, because the window is behind Roblox exactly
when you are using it. Start a run or a recording from the window and it brings
Roblox forward and counts down three seconds first — otherwise the click that
focused the game would be the first step of your build order.

The rest of this page is the command line, which does all of the same things.

## Why it can actually drive Roblox

Every "send keys to the window" recipe fails here, because Roblox ignores
`PostMessage` and `SendMessage`: a window message that never came up through
the input stack is not input as far as it is concerned. This uses **SendInput**,
which goes in at the same place the keyboard driver puts a keystroke, so Roblox
cannot tell the difference. Two details past that matter, and both are handled:

- **scan codes, not virtual keys.** A real keyboard puts a scan code on the
  wire; injectors that send only a virtual-key code are what games learn to
  distrust. `winput.py` sends the scan code with `wVk` left at zero.
- **relative mouse moves for the camera.** The camera reads raw deltas, not
  where the cursor is, so `SetCursorPos` turns nothing. Clicking a button is
  the opposite — that *is* cursor position — so clicks move the cursor and
  press, and camera work is a relative move.

`python macro.py check --live` proves it end to end: it focuses Roblox and makes
your character jump.

## Five minutes to a working farm

```powershell
cd C:\Users\bachk\roblox\KittyMacro

python macro.py check --live      # does input land in Roblox?  (jumps twice)
python macro.py record build      # F7, play one round by hand, F7
python macro.py play build        # watch it do that round back
python macro.py calibrate         # hover each button, F6 to capture
python macro.py run               # farm — F7 to start, F7 pause, F8 stop
```

### 1. Record your build order

`record build` watches the physical mouse and keyboard. Press **F7**, then play
one round exactly as you would: pick a unit, click the tile, upgrade, all of it.
Press **F7** again to finish. Every click and key press is saved with the gap
before it, so the timing of your own run is the timing of the macro's.

The recording is plain JSON in `config/routines/build.json`. Editing it by hand
is expected — nudge a coordinate, stretch a wait, delete a misclick.

Two routines, if you want them:

| Routine | When it runs | What belongs in it |
|---|---|---|
| `build` | once, at the start of a match | placing units, opening upgrades |
| `loop` | over and over until the match ends | upgrade clicks, ability keys |

Record the second one as its own short routine and point `routines.loop` at it
in `config/animevanguards.json`.

### 2. Calibrate the buttons around the match

`calibrate` asks for up to five screen points. Hover each one and press **F6**,
or **F8** to skip anything your setup does not have:

| Point | What to hover |
|---|---|
| `close` | the X on whatever popup greets you. Skip if none |
| `start` | the button that starts a match |
| `end_probe` | **a pixel that is only that colour when the results screen is up** |
| `replay` | Replay / Play Again on the results screen |
| `confirm` | a second confirmation click, if there is one |

`end_probe` is the one that matters. Put it on something loud — the coloured
header of the rewards panel, a button's fill — never on grey background that
looks the same mid-match. Get it right and the loop runs for hours; leave it
unset and the macro falls back to a fixed timer, which cuts off slow runs and
wastes the tail of fast ones.

Capture the results-screen points at the end of an actual run, while the panel
is on screen. `python macro.py mouse` prints the cursor position and the colour
under it live, if you would rather read them off and type them into the config.

### 3. Run it

```powershell
python macro.py run          # loop matches until you stop it
python macro.py run --once   # one match, then stop
```

It starts **paused**. Focus Roblox, press **F7**, and it goes.

## Playing a falling-notes event

The other macro plays the piano events — notes falling down a few lanes towards
a line, and you press that lane's key as one crosses it. Pick **Piano** in the
sidebar, then open the Piano tab.

1. **Type the keys**, one per lane, left to right: `asdfg` for a five-key
   event, or whatever yours uses. Any number of lanes works.
2. **Pick any one point on the track** — anywhere on it, roughly. It only needs
   to know which part of the screen to search.
3. **Press Find the lanes.** It watches a band of the screen around that point
   for five seconds and asks which columns changed. A lane changes as notes go
   through it; a divider, a key and the background do not. It tries dozens of
   rows and keeps the **lowest one where every lane is visible at once** —
   lowest because that is nearest the line you are hitting, and every lane
   because a keyboard hides some lanes behind its taller keys well before it
   hides the rest. Then it learns their colours and picks a threshold without
   being asked.
4. **Press F7.** You are already in the game.

It shows you what it saw, which is worth reading:

```
lowest row where all 5 lanes are visible: y=818
                            ████   ████  ████   ████  ████
                              ^     ^      ^     ^      ^
  670–730 centre 700 moved 175
  ...
  a: track (6, 19, 43), note (36, 57, 150) at 107 away, covered 25%
threshold 54 — noise reaches 0, the faintest note is 107 away
```

Five clusters for five keys, all reporting a similar track colour, is what
right looks like. Five different track colours — whites and greys among them —
means the points are on dividers or keys rather than on the lanes.

If you would rather place them yourself, pick both ends and the lanes in
between are spaced out evenly, or use the **pick** button under each lane to
place one at a time. Put them **on the track where the notes fall**, above any
key or pad that lights up when pressed — see the troubleshooting below for why
that one matters more than it sounds.

The Δ numbers on the Piano page are for checking its work afterwards, or for
setting the threshold by hand. **Trace 5s** prints what each lane actually
showed, run by run — track, note, or something else — which is how you find out
why a lane misbehaves instead of guessing.

### What it does with that

One `BitBlt` of the strip across every lane, about 180 times a second, and each
lane's pixel compared to its empty track. Reading the lanes one at a time with
`GetPixel` would cost five milliseconds each — forty for eight lanes, longer
than the game takes to draw two frames, with the eight answers coming from
eight different moments. For something falling that is not a detail, it is the
whole problem.

The key goes **down when the lane stops looking empty and up when it looks empty
again**. That taps the short notes and holds the long ones without either being
a special case.

### When it plays badly

**Hits early** — raise `offset` (milliseconds between seeing a note and pressing
it). This is the usual one: the top edge of a note reaches the sampler before
its middle does.

**Hits late** — move the sampling points a little further up the lane by
re-picking the two ends. Offset cannot go backwards in time.

**Misses notes in fast runs** — lower `gap`. It is the shortest allowed spacing
between two notes in one lane, and it is there to stop a flicker counting twice.

**Some lanes work and others do not** — the sampling points are not all on the
lanes. This is the common one, and **Find the lanes** exists to end it: it puts
every point where the picture actually moves, so no lane can end up on a
divider. The tell, if you placed them by hand, is five lanes reporting five
different track colours after a watch.

**Put the points on the falling-note track, not on the keys.** A key or pad that
lights up when pressed is the macro watching its own hand: it presses, the pad
lights, the pad never goes back to looking empty, and the key is never let go.
Somewhere up the lane where notes are still travelling has none of that
problem.

**A hold note never lets go** — the lane is not going back to looking like
track. Press **Trace 5s** while it plays: if the lane shows a third colour that
is neither track nor note, that is what it is stuck on. Move the point up the
lane, or turn **Match note** on to go by the note colour instead. `Max hold` is
the backstop — it lets go after that many milliseconds whatever the pixel says,
so one confused lane cannot sit on a key for the rest of the song.

**One lane never fires, and the log said it was "covered 60% of the time"** —
that lane spent most of the watch under a note, so there was no majority to
learn from. Watch again over a stretch of the song with gaps in it. If the
other lanes are fine, it usually fixes itself: a lane whose note colour is
everyone else's background gets turned back round automatically.

**Fires with nothing there** — raise `threshold` first. If the lane background
is animated rather than flat, matching a note colour works better than measuring
distance from an empty one: put the note's colour in `config/piano.json` as
`"note": [r, g, b]` on that lane and it will look for that instead.

**Holds when it should tap** — turn `hold notes` off, and every note becomes a
tap of `min hold` milliseconds.

**A long note stutters, or comes out short** — a hold is let go `offset` after
the note leaves the sampler, the same delay its press got, so the length is
right rather than short by an offset. `Let go` is the patience on top of that:
a seam, a beat marker or a sparkle crossing the sampling point reads as the
note ending, and letting go for one scan and pressing again breaks the hold.
Anything that comes back inside that window never happened. Raise it if holds
still stutter; lower it if the note *after* a hold gets swallowed.

**The lanes are not evenly spaced** — edit the `at` values in
`config/piano.json` directly. Two ends and even spacing is the quick path, not
the only one. A lane with `"do": "click"` clicks its own point instead of
pressing a key, for events played with the mouse.

## Farming the whole event

Playing one song well is half of it. The other half is that nobody wants to sit
there clicking Play forty times. The **Event** page is that other half, and
every part of it is a point you pick once — same as the lanes, for the same
reason: your resolution, your UI scale, your menus.

### When is a song over?

Three signals. The best of them is the one the game already draws for you.

**The song timer stopped counting.** If there is a clock on screen, point at
it, and the question is settled: numbers count while a song runs and stop when
it ends. Nothing to match, no threshold, no colour to reason about — the macro
only asks whether the pixels changed. **Watch it** on the Event page says
whether it is ticking right now, which is the whole check. Where a timer is
picked it is the only signal consulted, because a quiet stretch of chart stops
the track moving but does not stop the numbers counting.

The other two are for events without a clock.

**The track stops changing.** Notes falling is the only thing moving on that
strip, so when it goes still for a few seconds the song is done. The trap this
replaced was asking whether the lanes looked like *empty track*: when a song
ends, the panel that comes up is not empty track either, so "nothing looks like
the background" stayed true forever and every lane sat on its key. What
actually stops at the end of a song is change, so that is what is measured, and
a track that has held still for a second gets its keys let go of straight away
rather than waiting for the verdict.

**A pixel that only the results screen paints.** Pick one on the Event page — a
coloured header or button on the panel, not grey background — and a song ends
the moment that panel appears. Optional, and fiddlier than the timer.

Neither counts until the song has actually **started**, which is the other half
of the same idea: a menu, a loading screen and the song picker are all
perfectly still, so a macro armed before the music would otherwise decide the
song was over before it began — and then stop. A song has started when the hit
line has moved a dozen times inside three seconds; one cursor going past does
not do that, and a chart cannot help doing it. `not before` is counted from
there, so arming the macro and then taking your time in the menus is fine.

### The way in

A route is a list of steps, each one a point you picked:

```
Events  →  Special Events  →  scroll to the bottom  →  the Piano event  →  Play
```

Pick a step, press **Walk it**, watch where it lands, pick the next. A step
with nothing picked is skipped and said so in the log, so a half-built route
still walks as far as it goes — which is how you build one.

**Every click in a route is checked.** Roblox drops clicks — one that lands
while the UI is still animating never happened as far as the game is concerned,
and then the rest of the route clicks into empty space and the run is wasted.
So each one takes a coarse picture of the screen after the cursor arrives and
another after the button comes up, and presses again if nothing moved. Three
goes, then it carries on and says so.

It waits for the reaction rather than glancing once: a button that opens a menu
changes the screen immediately, but one that loads something takes a moment,
and deciding *there* that the click was dropped is worse than not checking at
all — the clicks that follow land wherever the game has got to by then.

How much movement counts is measured, not assumed. Two pictures before touching
anything say what the screen does on its own — almost nothing in a menu, a good
deal in a lobby full of other people's characters — and the click has to beat
that by a distance. Where the screen is so busy that no click could be told
apart from it, it says so and clicks once. A retry also takes the cursor off
the button and back before pressing, because some UI wants a fresh hover and
ignores everything else.
The clicks that cycle a difficulty are exempt — the change they make is far too
small to see, and one counted twice would pick the wrong difficulty. `+ click`,
`+ scroll`, `+ key` and `+ wait` add steps if your menus need a different
shape, `✕` drops one, and the seconds after each step are yours to set.

The second route, **back to the list**, is how it gets off the results screen
and onto the songs again. Both are also walkable from the command line:

```powershell
python macro.py walk into
python macro.py walk back
```

### Forty runs

Ten songs at four difficulties. **One point per row of the song list** — pick
them, `+ song` for another, or pick the first and the last and press **Space
them out** to fill the middle evenly. Your own click on a row beats arithmetic,
so picked rows are never overwritten.

Then the **difficulties**, which are a point and a number of clicks each. That
covers both shapes these events come in:

| | clicks |
|---|---|
| four separate buttons | one click each, on four different points |
| one button that cycles easy → medium → hard → expert | 0, 1, 2, 3 clicks on the same point |
| ditto, but the panel remembers where you left it | 0, 1, 1, 1 |

**One button that cycles** sets the middle row of that table up in one press.
Pick **Play** as well if the song panel has its own.

If all ten do not fit on screen, say how many do and it winds the list back to
the top and down by whole screenfuls — always from the top, because a list
already scrolled to somewhere unknown cannot be moved by a relative amount and
land anywhere knowable.

Last, **where a finished song leaves you**. Some events drop you back on the
song list; others throw you out of the event altogether, and then the way in
has to be walked again before every single run. That is slower and it is simply
what some events do, so it is a choice on the page rather than an assumption.

Then **Full session** on the Home page, and START:

```
song 3 hard — run 11 of 40
  song 3 at 587,288
  hard: 2 clicks at 1315,700
song 3 hard is under way
song 3 hard: the results screen came up after 2.4 min — 486 notes, 181 scans a second
  back to the list —
  1. leave the results screen
```

The tally lives in `config/piano_progress.json`, written after every run, so
stopping for dinner, closing the window or a crash all resume at song 7 expert
rather than song 1 easy. **Forget them** on the Event page starts the grid
over. Runs that ended on the clock rather than on an ending are listed at the
end, because those are the ones worth replaying by hand.

The same thing from the command line:

```powershell
python macro.py run --game piano --session
```

**Song after song** is the lighter version: it plays whatever is in front of
it, notices the song ending, waits for the next one to start, and plays that
too. No picking required, and you drive the menus.

## Keys

| Key | |
|---|---|
| `F7` | start / pause |
| `F8` | stop (and, while recording, discard) |
| `F6` | capture the point under the cursor, during `calibrate` |

They are polled globally, so they work while Roblox has focus. Change them in
`config/animevanguards.json` under `hotkeys` if a game binds them.

## It never types into the wrong window

The loop only sends input while the Roblox window is in front. Alt-tab away and
it lets go of every key and button it was holding and waits; alt-tab back and it
carries on. Stopping does the same. That is the one rule the whole thing is
built around — a macro that keeps clicking into your browser is worse than none.

## Settings

`config/animevanguards.json`, written on first run:

| | |
|---|---|
| `routines.build` / `.loop` | which recordings to use |
| `routines.loop_every` | seconds between replays of the loop routine |
| `routines.speed` | scales every recorded gap. Above 1.0 outruns your income |
| `match.start_delay` | after pressing start, before the build order |
| `match.max_seconds` | give up on a match that never ends |
| `match.load_seconds` | after replay, before the next match starts |
| `tolerance` | per-channel colour slack on `end_probe`, 0–255 |
| `antiafk` | a jump every few minutes so Roblox does not idle you out |

## When it does not work

**`sendinput REFUSED`** — Roblox is running as administrator and this is not.
Windows blocks input from a lower integrity level. Run the macro elevated too.

**Clicks land in the wrong place** — the coordinates are absolute screen pixels,
so they are tied to the resolution, the window position and the UI scale you
recorded at. Change any of those and you re-record. Run Roblox windowed or in
its own borderless fullscreen and leave the window where it is.

**`end_probe` never fires** — the colour under that pixel is not what it was.
Re-capture it with the results screen up, or raise `tolerance`. If pixel reads
come back black, something is drawing over the desktop; a game in exclusive
fullscreen cannot be read this way (Roblox does not use it, but overlays can).

**`roblox NOT FOUND`** — it matches the window titled `Roblox`. Roblox Studio is
deliberately excluded.

**Nothing happens at all and no error** — the macro is paused, or Roblox is not
the front window. Both say so, in the console and in the window's header.

**`No module named tkinter`** — the window needs Tk, which comes with the
installer from python.org and is left out of some slimmed-down builds. The
command line does not need it.

## Layout

```
macro.py                      the command line
gui.py                        the window
KittyMacro.bat                double-click, and the window opens
kittymacro/winput.py          SendInput: keys, clicks, relative mouse
kittymacro/screen.py          one-pixel reads, whole-strip reads, and
                              finding the Roblox window
kittymacro/hotkeys.py         F6/F7/F8 while the game has focus — and the
                              window's buttons, arriving the same way
kittymacro/runner.py          pause, stop, focus gate, release-everything
kittymacro/routine.py         recording and playback
kittymacro/config.py          settings and routines on disk
kittymacro/macros/            one module per game
  animevanguards.py           run a stage, place units, take the reward, again
  piano.py                    read the hit line, press what crosses it
config/                       your settings and recordings
  piano_progress.json         which of the forty runs are done
```

Adding another game is one module in `macros/` with `DEFAULTS`, `calibrate()`
and `run()` — everything above it is game-agnostic.