This article is ratedStart-class on Wikipedia'scontent assessment scale. It is of interest to the followingWikiProjects:

StatisticsLow‑importance This article is within the scope ofWikiProject Statistics, a collaborative effort to improve the coverage ofstatistics on Wikipedia. If you would like to participate, please visit the project page, where you can jointhe discussion and see a list of open tasks.StatisticsWikipedia:WikiProject StatisticsTemplate:WikiProject StatisticsStatistics Low This article has been rated asLow-importance on theimportance scale. MathematicsLow‑priority This article is within the scope ofWikiProject Mathematics, a collaborative effort to improve the coverage ofmathematics on Wikipedia. If you would like to participate, please visit the project page, where you can jointhe discussion and see a list of open tasks.MathematicsWikipedia:WikiProject MathematicsTemplate:WikiProject Mathematicsmathematics Low This article has been rated asLow-priority on theproject's priority scale. PhysicsLow‑importance This article is within the scope ofWikiProject Physics, a collaborative effort to improve the coverage ofPhysics on Wikipedia. If you would like to participate, please visit the project page, where you can jointhe discussion and see a list of open tasks.PhysicsWikipedia:WikiProject PhysicsTemplate:WikiProject Physicsphysics Low This article has been rated asLow-importance on theproject's importance scale.

This article is or was the subject of a Wiki Education Foundation-supported course assignment. Further details are availableon the course page. Student editor(s):Chrishaack.

Above undated message substituted fromTemplate:Dashboard.wikiedu.org assignment byPrimeBOT (talk)18:42, 16 January 2022 (UTC)[reply]

This article uses the notation KL(p, q) and also DKL(p || m) when talking about Kullback-Leibler divergence. Are these notations two ways of expressing the same idea? If so, the article may want to indicate this equivalence.

The log-likelihood of the training data for a multinomial model is the same as the cross-entropy of the data. (Elements of Statistical Learning, page 32)

L(theta) = sum (all classes k) I(G=k) log Pr(G=k | X = x)

I guess "I(G=k)" is p and Pr(G=k | X=x) is q here.

Could somebody in the know please add this? Thanks!

This article was automatically assessed because at least one WikiProject had rated the article as stub, and the rating on other projects was brought up to Stub class.BetacommandBot09:46, 10 November 2007 (UTC)[reply]

To improve clarity, this stub should be merged into the "Kullback–Leibler divergence" article. "Cross entropy" and "relative entropy" refer to the same quantity in the literature, at least up to a sign convention.DRB (talk)00:39, 15 January 2010 (UTC)[reply]

Hmmm... fair idea but not a cut-and-dry case. maybe put a merge tag on it, at least.Kevin Baas^talk14:07, 15 January 2010 (UTC)[reply]

At least one analogy or example would be most helpful. I get no sense at all as to what this article is attempting to describe.Marc W. Abel (talk)02:24, 7 May 2012 (UTC)[reply]

In the text, "cross-entropy" is written with a hyphen every time (as I believe to be correct). In the title of the page, however, it is not hyphenated. Why is this?— Precedingunsigned comment added byThomas Tvileren (talk •contribs)12:48, 27 May 2022 (UTC)[reply]

I thought that was odd too. I've moved the page to fix it. —The Anome (talk)12:23, 7 August 2023 (UTC)[reply]

A title that's 12 words long seems kind of excessive. Also, it reads somewhat like an informal research paper. Along with the fact that I'm not really an editor, I don't know anything about this topic and so I'm not sure if any edits I make would actually improve it. I also don't know what tags would be helpful to add, if I'm even qualified to do so. Maybe a cleanup or copyedit tag?dr1zzled_d1ce(talk)19:30, 23 June 2024 (UTC)[reply]

• Start-Class Statistics articles
• Low-importance Statistics articles
• WikiProject Statistics articles
• Start-Class mathematics articles
• Low-priority mathematics articles
• Start-Class physics articles
• Low-importance physics articles
• Start-Class physics articles of Low-importance