utility function

A function that assigns a real number to each outcome or state of the world so that higher numbers represent more preferred outcomes. By the von Neumann–Morgenstern theorem, any agent whose preferences over gambles are complete, transitive, continuous and independent acts as if maximizing the expected valueexpected value, n.: The long-run average you would get if a chancy situation could be repeated many times — 3.5 for a fair die, even though no single roll shows 3.5. of some utility function, unique up to a positive affine transformation (rescaling plus a shift). Expected-utility maximization is the standard model of rational choice in economics and of intelligent agentsintelligent agent, n.: In AI theory, a decision-maker, such as a robot, a self-driving car, a chess program or a helpful piece of software, that notices what is going on around it and responds in ways that serve its goals. in classical AI, where a reward functionreward function, n.: The rule that decides how many points an AI agent earns for each situation and action — in effect, the written-down definition of the task it is being trained to do. or objective functionobjective function, n.: The score that an optimization process tries to push as low, or as high, as possible. For an AI, it defines what 'doing well' means during training. plays the role of utility. In AI alignmentAI alignment, n.: Making an AI system genuinely try to do what the people it serves want — not a literal-minded, distorted or self-serving version of it. Also the research field working on this., utility functions frame central debates: misspecified utilities lead to specification gamingspecification gaming, n.: When an AI meets the letter of its goal while defeating its purpose, like a genie who grants exactly the words of a wish instead of what the wisher meant.; coherence arguments and instrumental convergenceinstrumental convergence, n.: The idea that almost any goal is easier to reach if you stay switched on, keep your goal and gain resources and power, so very different AIs might all seek those same things. suggest that capable agents behave like expected-utility maximizers with incentives for power-seekingpower-seeking, n.: The tendency of a goal-pursuing AI to gather resources, influence and options — and to resist being stopped — because these help with almost any goal it might have.; critics reply that real agents, including language modelslarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code., need not have coherent utility functions at all; and value learningvalue learning, n.: Teaching an AI what people want by having it learn from their behavior, choices and feedback, rather than trying to write every goal down in advance. tries to infer human utilities from behavior. Prospect theory and other non-expected-utility models describe actual human choices better.

If the utility function counts only tickets closed, the agent will close tickets without fixing anything.