MyComponent does not exist, so it should not be listed in the components of your app. Update your app code to:
new Vue({
el: '#app',
components: {
CustomComponent, // Not used; template passes a string instead
RenderComponent
},
template: '<render-component component="CustomComponent"></render-component>'
})
Check out this fiddle for reference.
Explanation
You correctly registered CustomComponent as MyComponent in the main app component and can use it in the template directly. However, you are also passing a string with the value "MyComponent" to the RenderComponent component. This causes RenderComponent to attempt instantiating a MyComponent, which is not found in the global scope. It exists only within the main app component where it was registered.
If you try using a binding (:component) instead of passing MyComponent, it still won't work because you cannot pass components as props.
(EDIT: You can do it in an unconventional way like this: :foo="$options.components.foo". Refer to this comment from Evan You)
Interesting Alternative
You mentioned attempting to pass it from data, and while it may work, I cannot confirm if it's a good practice. It might become more complex when migrating to single-file components without access to a global scope. Despite being confusing, this method works:
new Vue({
el: '#app',
data: {
MyComponent: CustomComponent // Retrieved from global scope
},
components: {
// CustomComponent, // Omitted in this version
RenderComponent
},
template: '<render-component :component="MyComponent"></render-component>'
})
View the updated fiddle here.