Back in the days of C and no templates... we had void pointers. Good thing they weren't made obsolete, because you have just seen that templates aren't omnipotent gods. This situation is perfect for some void pointer hacks.

Here, I tried to make an example that was as simple as possible:

		struct ResourceManager {
			
			typedef std::function<void* ()> TFactoryFunctor;
			
			std::map<std::string, TFactoryFunctor> m_factories;
			
			void subscribe (std::string key, TFactoryFunctor factory) {
				m_factories[key] = factory;
			}
			
			template <class T>
			std::shared_ptr<T> load (std::string key) {
				auto& factoryFunctor = m_factories.at(key);
				void* ptr = factoryFunctor();
				
				return std::shared_ptr<T>(static_cast<T*>(ptr));
			}
			
		};

		struct ResourceFactory {
			
			std::string m_path;
			
			void setFilePath (std::string path) {
				m_path = path;
			}
			
			std::string getFilePath() const {
				return m_path;
			}
			
			virtual void* load() =0;
			
			void* operator() () {
				return this->load();
			}
			
		};

		struct TextureFactory : ResourceFactory {
			
			std::shared_ptr<Texture> m_texPtr;
			Texture* m_tex;
			
			TextureFactory (std::string fileName) {
				setFilePath(TEXTURES_FOLDER + fileName);
			}
			
			void* load() override {
				if (!m_texPtr) {
					m_tex = new Texture();
					m_tex->loadFromFile(getFilePath());
					
					m_texPtr = std::shared_ptr<Texture>(m_tex);
				}
				
				return m_tex;
			}
			
		};

		// ---

		ResourceManager daMan;

		void subscribeTextures() {
			daMan.subscribe("hero", TextureFactory("hero.png"));
			daMan.subscribe("foe", TextureFactory("foe.png"));
			daMan.subscribe("tileset", TextureFactory("tileset.png"));
		}
		
		void loadHeroTexture (Hero& hero) {
			auto texPtr = daMan.load<Texture>("hero");
			
			hero.claimOwnership(texPtr); // Maybe you want to keep track of who's using the resource?
			hero.sprite.setTexture(*texPtr);
		}

I didn't test this code to make sure that it's functional and I'm not sure whether it's a good idea to manipulate a raw pointer that's stored in a `shared_ptr` in the way within `TextureFactory::load`, but I hope you get the idea.

In a different, more object-oriented language you would probably use `object` (or some other more specialized base class) and downcasts instead of `void*`, but C++ doesn't have such luxuries. Maybe you can make your own `ResourceBase` base class? You'd have to make _some_ wrappings, do so at your own expenses.