SN-15 is using a centralized scoring mechanism and probably gaming

burn police
1

seems like they are using a centralized scoring mechanism, validators are hitting an endpoint and getting the score. I tried it with a top and a bottom key and both seem to return similar scores with a couple of tries. not decentralized at all and easy to game but not sure if they are doing it.

I think everything needs to happen in the validator codebase all the time, not like this. if it’s centralized like this, it needs an explanation.

you can play with it with different hotkeys from the metagraph from top to bottom:

curl -X POST https://quant-api.opengradient.ai/api/subnet/evaluate \
  -H "Content-Type: application/json" \
  -d '{
    "quant_query": {"query": "What is the current price of Bitcoin?", "userID": "test", "metadata": {}},
    "quant_response": {
      "response": "I checked the latest data and Bitcoin is currently trading at $43,250 USD. The market has shown bullish momentum over the past 24 hours with a 3.2% increase. Trading volume is strong at $28B.",
      "signature": "",
      "proofs": [],
      "metadata": {"miner_id": "5GTCcQAXw2LTdJzHL8yN1Ttf7HvQSBhwUfWAXCnMn42219iC"}
    }
  }'
1 Like
2

@commander_hawk for absolute clarity please provide:

  • link to their validator repo which downloads the weights from the central api
  • output of running your script followed by quick analysis, so that we can see what you see
3

@rhef they mentioned that they will be moving the scoring mechanism to the validator code and merge it this week. I will follow the process and post here more details (like the ones you asked) if they don’t merge or if that won’t change how it works.

1 Like