While they serve a similar purpose to a traditional cabinet speaker (i.e. floor-standing, bookshelf, centre and satellite speakers)  a custom speaker is designed and engineered differently for its specific purpose.

A traditional speaker works as a total unit balancing their speaker drive unit(s), cross-over electronics and cabinet to offer the best 'in-room' sound reproduction. A custom speaker (flush mounted in-wall or in-ceiling)  is designed to be discrete and therefore having no (visible) cabinet and a discrete cosmetic design has specific challenges (heat and moisture among others) to overcome to ensure the best sound quality.

A good custom speaker is made of materials that offer high power handling, smooth audio response and extended bandwidth (good highs and lows). Low resonance and low distortion along with a tweeter which can be aimed towards the listener’s position are also beneficial features.

Some speakers are designed to be aimed towards the listener which is great for Home Theatre applications - also useful for unique interior applications such as vaulted ceiling designs. 

One of the primary reasons for their existence, is custom speakers are by design visibly discrete. Many manufacturers approach the industrial design to hide, as much as possible the speaker's location once installed. There are varying degrees of aesthetic subtly with the bezel (surrounding frame) and grill design becoming more and more discrete as the level (and price) increases. Some can be painted to colour-match the interior design. There are even designs that are completely invisible.