That API is not meant for extension. It is supposed to be a stable, consistent API on which you can rely between versions of Alfresco and different instances of Alfresco. Of course you can try to create a patch / PR and submit that to Alfresco for inclusion in the next iteration of the Public ReST API.
Technically you might be able to extend it by meddling with some of the Java classes in Alfresco, but that would defeat the purpose of the public API. You can always create your own web script to extend the ReST API with any specific support that you need.