Introduction
I have been learning C++ and wanted a real project to work on. The idea: a move tool for Maya that does collision detection against the rest of the scene in real time - no pre-setup, no simulation. You grab an object, push it around, and it stops at other geometry. Useful for set dressing and environment work.
Here’s where it’s at:
What follows is a snapshot of the current state. It’s not finished.
How it works
The goal was to handle scenes of any size. Workflow:
- When the tool activates, an empty Bullet physics world is created in the background.
- An R-tree gets built from the bounding boxes of every mesh in the scene. This is the spatial index for fast “what could I collide with” queries.
- When the user selects something, it becomes an active rigid body in the physics world.
- On drag,
doDragruns collision detection against whatever the R-tree flagged as a candidate.
R-tree for collision candidates
Every mesh in Maya has a bounding box. Cheap to check, and good enough as a first-pass filter - if two boxes don’t overlap, the meshes can’t collide.
I used Boost’s R-tree, keyed on the world-space bounding boxes of scene meshes. When the user drags an object, I query the tree with that object’s bounding box and only run real collision against the hits.
Here’s the doDrag flow in more detail:
R-tree init:
MStatus CollisionCandidatesFinder::initializeRTree()
{
if (this->allSceneMFnMeshes.empty()) {
MGlobal::displayError("MFnMeshes vector is empty");
return MS::kFailure;
}
MStatus status;
for (size_t i = 0; i < this->allSceneMFnMeshes.size(); ++i) {
MDagPath dagPath;
status = this->allSceneMFnMeshes[i]->getPath(dagPath);
if (status != MS::kSuccess) {
MGlobal::displayError("Failed to get MDagPath from MFnMesh");
return status;
}
MBoundingBox mbbox = this->allSceneMFnMeshes[i]->boundingBox();
MMatrix worldMatrix = dagPath.inclusiveMatrix();
MPoint minPoint = mbbox.min() * worldMatrix;
MPoint maxPoint = mbbox.max() * worldMatrix;
box bbox(point(minPoint.x, minPoint.y, minPoint.z), point(maxPoint.x, maxPoint.y, maxPoint.z));
this->rTree.insert(std::make_pair(bbox, i));
}
return MS::kSuccess;
}
Querying it:
box queryBox(
point(worldMinPoint.x, worldMinPoint.y, worldMinPoint.z),
point(worldMaxPoint.x, worldMaxPoint.y, worldMaxPoint.z)
);
std::vector<value> result;
this->rtree.query(bgi::intersects(queryBox), std::back_inserter(result));
This part worked well - fast and responsive, even in denser scenes.
What broke
Converting an MFnMesh to a btRigidBody is expensive. You iterate every triangle, and doing that on demand for each potential collider mid-drag was too slow for real-time.
So I dropped the dynamic R-tree approach. Now all scene meshes get converted to Bullet shapes once, at tool startup, and those shapes are used to build both static and active rigid bodies. This caps the tool to small/medium scenes for now, but it’s responsive. Updated workflow:
Maya mesh to Bullet shape
Static colliders (everything not being dragged) use btBvhTriangleMeshShape built from the full triangle list. Active rigid bodies (the current selection) use a convex hull - cheaper, and good enough for dragging:
btCollisionShape* BulletCollisionHandler::convertMFnMeshToStaticCollisionShape(MFnMesh* mfnMesh) {
// Get the points in world space
MPointArray mayaVertices;
mfnMesh->getPoints(mayaVertices, MSpace::kWorld);
// Create the Bullet triangle mesh
btTriangleMesh* triMesh = new btTriangleMesh();
// Get triangles from the mesh
MIntArray triangleCounts, triangleVertices;
mfnMesh->getTriangles(triangleCounts, triangleVertices);
// Index variables for triangleVertices
int triangleIndex = 0;
for (unsigned int i = 0; i < triangleCounts.length(); ++i) {
for (int j = 0; j < triangleCounts[i]; ++j) {
btVector3 vertices[3];
for (int k = 0; k < 3; ++k) {
// Get the vertex index
int vertexIndex = triangleVertices[triangleIndex + k];
// Transform the vertex position to world space
MPoint worldSpaceVertex = mayaVertices[vertexIndex];
vertices[k] = btVector3(
static_cast<btScalar>(worldSpaceVertex.x),
static_cast<btScalar>(worldSpaceVertex.z),
static_cast<btScalar>(-worldSpaceVertex.y)
);
}
// Add the triangle to the mesh
triMesh->addTriangle(vertices[0], vertices[1], vertices[2]);
// Move to the next set of vertices
triangleIndex += 3;
}
}
// Create the mesh shape
bool useQuantizedAABBCompression = true;
btBvhTriangleMeshShape* meshShape = new btBvhTriangleMeshShape(triMesh, useQuantizedAABBCompression);
return meshShape;
}
btCollisionShape* BulletCollisionHandler::convertMFnMeshToActiveCollisionShape(MFnMesh* mfnMesh) {
// Get the points in local space
MPointArray mayaVertices;
mfnMesh->getPoints(mayaVertices, MSpace::kObject);
// Create the Bullet convex hull shape
btConvexHullShape* convexHull = new btConvexHullShape();
// Loop through the vertices and add them to the convex hull
for (unsigned int i = 0; i < mayaVertices.length(); ++i) {
MPoint vertex = mayaVertices[i];
btVector3 bulletVertex(
static_cast<btScalar>(vertex.x),
static_cast<btScalar>(vertex.z),
static_cast<btScalar>(-vertex.y)
);
// Add the vertex to the convex hull shape
convexHull->addPoint(bulletVertex);
}
convexHull->optimizeConvexHull();
convexHull->initializePolyhedralFeatures();
return convexHull;
}
Singletons for state
Maya recreates the manipulator on every reselection. I didn’t want to rebuild every collider each time, so CollisionCandidatesFinder and BulletCollisionHandler are singletons:
class CollisionCandidatesFinder {
public:
/**
* @brief Retrieves the singleton instance of the class.
* @return Reference to the singleton instance.
*/
static CollisionCandidatesFinder& getInstance();
// Prevent copying and assignment.
CollisionCandidatesFinder(const CollisionCandidatesFinder&) = delete;
CollisionCandidatesFinder& operator=(const CollisionCandidatesFinder&) = delete;
private:
CollisionCandidatesFinder(); // Constructor is private for singleton
~CollisionCandidatesFinder(); // Destructor
};
Dragging
I drive the active rigid body with linear velocity, computed from the delta between the manipulator’s position and the body’s position. The interesting bit is clamping - if the user drags fast, the velocity vector blows up and Bullet’s collision response falls apart. So I clamp to a sane range and zero out anything below a small threshold:
// object pos of locator
MPoint currentPosition;
this->getConverterManipValue(0, currentPosition);
btVector3 currentPos(
this->currentManipPosition.x,
this->currentManipPosition.z,
-(this->currentManipPosition.y)
);
btVector3 targetPos(
currentPosition.x,
currentPosition.z,
-currentPosition.y
);
// Here we calculate velocity vector by clamping in specified range
// it is nessesary to do in order to avoid too huge or too small velocity values
float timeStep = 1.0f / 60.0f;
btVector3 requiredVelocity = (targetPos - currentPos) / timeStep;
requiredVelocity *= 0.04*2;
float threshold = 0.05f;
if (std::abs(requiredVelocity.x()) < threshold) requiredVelocity.setX(0);
if (std::abs(requiredVelocity.y()) < threshold) requiredVelocity.setY(0);
if (std::abs(requiredVelocity.z()) < threshold) requiredVelocity.setZ(0);
// Define range for clamping
float minValue = -0.4f*4;
float maxValue = 0.4f*4;
// Clamp values within the range
requiredVelocity.setX(std::min(std::max(requiredVelocity.x(), minValue), maxValue));
requiredVelocity.setY(std::min(std::max(requiredVelocity.y(), minValue), maxValue));
requiredVelocity.setZ(std::min(std::max(requiredVelocity.z(), minValue), maxValue));
MVector avgPosition(0.000, 0.000, 0.000);
unsigned int count = 0;
for (auto& pair : this->bulletCollisionHandler.activeRigidBodies) {
std::string name = pair.first;
btRigidBody* body = pair.second;
body->setLinearVelocity(requiredVelocity);
}
What’s missing
- No pivot support on the selection.
- No rotate or scale - the standard Maya hotkeys don’t bind to this tool yet.
- Stability needs work.
- No USD support, limited mesh type coverage.
- No profiling, no unit tests, memory leaks haven’t been checked.
- Performance on large scenes is unsolved; the startup conversion is the bottleneck.
Plenty to do. Source is on GitHub.