NEOLearn

Picking

Make meshes, sprites and UI respond to the mouse and touch with pointer events such as PointerClick and PointerDrag, observed on the entity.

Updated October 20, 2026

On this page

Picking answers "which entity is under this pointer?" for every mouse and finger, and turns the answer into events on that entity: hovered, pressed, clicked, dragged, dropped. You attach observers to the entities you care about, the same way for a 3D mesh, a sprite or a UI node.

Click a cube#

src/main.rs
use neo::prelude::*;

fn main() {
    App::new()
        // mesh picking isn't one of the default plugins
        .add_plugins((DefaultPlugins, MeshPickingPlugin))
        .add_systems(Startup, setup)
        .run();
}

fn setup(
    mut commands: Commands,
    mut meshes: ResMut<Assets<Mesh>>,
    mut materials: ResMut<Assets<StandardMaterial>>,
) {
    commands.spawn((Camera3d::default(), Transform::from_xyz(0.0, 3.0, 6.0).looking_at(Vec3::ZERO, Vec3::Y)));
    commands.spawn((PointLight::default(), Transform::from_xyz(3.0, 6.0, 3.0)));

    commands
        .spawn((
            Mesh3d(meshes.add(Cuboid::new(1.0, 1.0, 1.0))),
            MeshMaterial3d(materials.add(Color::srgb_u8(124, 144, 255))),
        ))
        .observe(|click: On<PointerClick>| info!("clicked {} with {:?}", click.entity, click.button))
        .observe(|drag: On<PointerDrag>, mut transforms: Query<&mut Transform>| {
            if let Ok(mut t) = transforms.get_mut(drag.entity) {
                t.rotate_y(drag.delta.x * 0.02);
            }
        });
}

The pieces#

PieceWhat it does
PointersThe mouse (PointerId::Mouse) and each finger (PointerId::Touch(id)). Custom pointers are possible too.
BackendsHit tests that say what each pointer is over. UI and sprite picking are added by the UI and sprite plugins. Mesh picking needs MeshPickingPlugin, which you add.
Hover and eventsThe engine sorts the hits by camera order and depth, works out what's hovered, and triggers pointer events.
Your observers.observe(...) on an entity, or a global add_observer, reacts to the events.

NEO's builds include all three backends (mesh, sprite and UI picking) on every platform, including the web.

Pointer events#

Every event has entity (the target) and pointer (its id, render target and position in the window). Most also carry hit, the HitData with the camera, depth, and the world position and normal where the backend knows them.

GroupEvents
HoverPointerOver, PointerOut (the pointer moved onto or off this entity), PointerEnter, PointerLeave (like the web's mouseenter: sent to the entity and the ancestors it newly entered), PointerMove.
PressPointerPress, PointerRelease, PointerClick (with button, duration and count for double clicks), PointerCancel.
DragPointerDragStart, PointerDrag (with delta and distance in screen pixels), PointerDragEnd.
DropPointerDragEnter, PointerDragOver, PointerDragLeave, PointerDragDrop on the entity under the dragged one.
ScrollPointerScroll with the wheel's x, y and unit.

Bubbling#

Pointer events bubble: an event on a child is then sent to its parent, then the parent's parent, up to the window. That's what lets one observer on a UI panel handle clicks on all its children. Stop it in an observer with event.propagate(false):

Rust
commands.spawn(Node::default()).observe(|mut click: On<PointerClick>| {
    click.propagate(false); // the parents don't get this click
});

Pointer events are also messages, so a system can read them with MessageReader<PointerClick> instead of observing. See Observers for how observers and propagation work.

Choosing what can be picked#

The Pickable component controls an entity:

FieldDefaultWhat it does
should_block_lowertrueStops the entities behind this one from being hovered. Set false for a see-through overlay.
is_hoverabletrueWhether this entity can be hovered and get events at all.

Pickable::IGNORE sets both to false: the pointer goes straight through. NEO's touch controls use it so the on-screen stick never blocks the game.

By default mesh picking tests every mesh, which is handy while you build. For a large scene, make it opt-in: insert MeshPickingSettings { require_markers: true, ..default() }, then add Pickable to the meshes and MeshPickingCamera to the cameras that should pick. Sprites test the opaque pixels of the sprite by default; SpritePickingSettings can switch to the bounding box.

Hover and press state#

To style something by its state instead of handling events, use these components:

ComponentWhat it holds
HoveredInsert it on an entity: Hovered(true) while a pointer is over it or its children. It only changes on enter and leave, so Changed<Hovered> is cheap.
PickingInteractionNone, Hovered or Pressed, across all pointers. The engine inserts it on entities that are hovered.

UI buttons also have Interaction; see Interaction.

Ray casts#

For gameplay hit tests that aren't tied to a pointer, such as a laser or line of sight, use the MeshRayCast system parameter:

Rust
fn laser(mut ray_cast: MeshRayCast, mut gizmos: Gizmos) {
    let ray = Ray3d::new(Vec3::new(0.0, 1.0, 5.0), Dir3::NEG_Z);
    if let Some((entity, hit)) = ray_cast.cast_ray(ray, &MeshRayCastSettings::default()).first() {
        info!("hit {entity} at {}", hit.point);
        gizmos.sphere(hit.point, 0.05, Color::WHITE);
    }
}

cast_ray returns every hit, nearest first. MeshRayCastSettings filters by visibility and by entity. NEO's character games use the character kit's collision, or the physics world and its ray casts, for gameplay; ray casts against render meshes suit tools and simple games.

Debugging#

DebugPickingPlugin shows what each pointer hits and logs events, but it's part of the developer tools, which aren't in NEO's default builds; see Developer tools.

Examples: cargo run --example simple_picking, mesh_picking, sprite_picking, dragdrop_picking, draggable_slider and custom_hit_data.

Next steps#