Hi @srenfrow,
WooCommerce does not have a default widget for filtering products by categories. The usual solution would be to use third-party widgets.
Another solution, which could be redundant, is to add use the categories as attributes, after which you can use the Filter by Attributes widget. This is not generally intended for filtering products by category, but it is a workaround if you don’t want to use plugins.