|
Synthetic vision
Synthetic vision is achieved in CoppeliaSim via vision sensors: they can perform offscreen rendering of the scene in different modes. The generated image can then be modified/processed in various ways, for instance from within a vision callback function:
image processing with OpenCV (via the image plugin)
simple image processing via the vision plugin
sim.transformImage
sim.transformBuffer
Following shows a vision callback function that illustrates 3 different ways to modify an acquired image:
function sysCall_vision(inData)
-- inData.handle: handle of the vision sensor
-- inData.resolution: resolution of the vision sensor
-- inData.clippingPlanes: near and far clipping plane of the vision sensor
-- inData.viewAngle: the view angle if the vision sensor is in perspective operation
-- inData.orthoSize: the size of the view if the vision sensor is in orthogonal operation
-- inData.perspectiveOperation: whether the vision sensor is in perspective operation
-- e.g. use very simple functions from the vision plugin (simExtVision):
simVision.sensorImgToWorkImg(inData.handle)
simVision.edgeDetectionOnWorkImg(inData.handle,0.2)
simVision.swapWorkImgWithBuffer1(inData.handle)
simVision.uniformImgToWorkImg(inData.handle,{0.5,0.25,0})
simVision.addBuffer1ToWorkImg(inData.handle)
simVision.workImgToSensorImg(inData.handle)
-- and/or, use more powerful functions from the image plugin (simExtImage):
if simIM then
local imgHandle=simIM.readFromVisionSensor(inData.handle)
local center={inData.resolution[1]/2,inData.resolution[2]/2}
local radius=(inData.resolution[1]+inData.resolution[2])/8
simIM.circle(imgHandle,center,radius,{255,255,0},4)
simIM.writeToVisionSensor(imgHandle,inData.handle)
simIM.destroy(imgHandle)
end
-- Or you can directly operate on the image buffer:
local image=sim.getVisionSensorImg(inData.handle)
sim.transformImage(image,inData.resolution,4)
image=sim.transformBuffer(image,sim.buffer_uint8rgb,1,0,sim.buffer_uint8bgr)
sim.setVisionSensorImg(inData.handle,image)
outData={}
outData.trigger=true -- whether the sensor should trigger
-- filters may append packets (in packed form, use sim.packFloatTable to pack) to this table:
outData.packedPackets={sim.packFloatTable({1,42,57})}
return outData
end
|