To be fair the context of that marx quote is specifically about the people during a revolution forming organised militia.

The quote gets used completely out of context a lot.

People should actually read the whole thing. Relevant excerpt:

There is no doubt that during the further course of the revolution in Germany, the petty-bourgeois democrats will for the moment acquire a predominant influence. The question is, therefore, what is to be the attitude of the proletariat, and in particular of the League towards them:

1. While present conditions continue, in which the petty-bourgeois democrats are also oppressed;
2. In the coming revolutionary struggle, which will put them in a dominant position;
3. After this struggle, during the period of petty-bourgeois predominance over the classes which have been overthrown and over the proletariat.

1. [snip]
2. To be able forcefully and threateningly to oppose this party, whose betrayal of the workers will begin with the very first hour of victory, the workers must be armed and organized. The whole proletariat must be armed at once with muskets, rifles, cannon and ammunition, and the revival of the old-style citizens’ militia, directed against the workers, must be opposed. Where the formation of this militia cannot be prevented, the workers must try to organize themselves independently as a proletarian guard, with elected leaders and with their own elected general staff; they must try to place themselves not under the orders of the state authority but of the revolutionary local councils set up by the workers. Where the workers are employed by the state, they must arm and organize themselves into special corps with elected leaders, or as a part of the proletarian guard. Under no pretext should arms and ammunition be surrendered; any attempt to disarm the workers must be frustrated, by force if necessary. The destruction of the bourgeois democrats’ influence over the workers, and the enforcement of conditions which will compromise the rule of bourgeois democracy, which is for the moment inevitable, and make it as difficult as possible – these are the main points which the proletariat and therefore the League must keep in mind during and after the approaching uprising.

He was essentially laying out what to do in each step. With step 2 being the very immediate building up of arms prior to revolution. Arming the people and building the arms required for revolution. This is quite different to "the people should be armed at all times" and it's a little inaccurate that the quote gets used that way.