• 0 Posts
  • 43 Comments
Joined 1 year ago
cake
Cake day: June 10th, 2023

help-circle

  • This product is so new that we have not yet determined the actual retail price. Whatever it turns out to be, you’ll certainly want at least one. Send us your actual credit card (not just the number!) along with a sample signature, and when the price has been finalized, we’ll charge your card accordingly.

    Order one and let us know!



















  • I’m not really following you but I think we might be on similar paths. I’m just shooting in absolute darkness so don’t hold much weight to my guess.

    What makes transformers brilliant is the attention mechanism. That is brilliant in turn because it’s dynamic, depending on your query (also some other stuff). This allows the transformer to be able to distinguish between bat and bat, the animal and the stick.

    You know what I bet they didn’t do in testing or training? A nonsensical query that contains thousands of one word, repeating.

    So my guess is simply that this query took the model so far out of its training space that the model weights have no ability to control the output in a reasonable way.

    As for why it would output training data and not random nonsense? That’s a weak point in my understanding and I can only say “luck,” which is, of course, a way of saying I have no clue.