Task #2448: Dashboard stats wrong - Infrastructure - The Document Foundation Redmine

Actions

#1

Updated by Guilhem Moulin over 6 years ago

Florian Effenberger wrote:

Xisco researched on the Dashboard (thanks!), and it seems there are quite some stats wrong:

Why git://gerrit.libreoffice.org/bibisect-win32-5.2, git://gerrit.libreoffice.org/bibisect-win32-5.1, git://gerrit.libreoffice.org/bibisect-win32-5.0 or git://gerrit.libreoffice.org/bibisect-macosx-64-5.0 are listed in the repositories section?

I mentioned that last August in the private marketing list (Message-ID: <20170802100347.2a2cszpmxcndoaye@localhost.localdomain>) but no one followed up on this so I just left the bibisect repos. I think they were added by the bitergia folks; if there is consensus that they should be removed I can do it, of course.

Guilhem, is this something you can fix?

Checking the other two points now.

Actions

Copy link

#2

Updated by Florian Effenberger over 6 years ago

I mentioned that last August in the private marketing list (Message-ID:
<20170802100347.2a2cszpmxcndoaye@localhost.localdomain
<mailto:20170802100347.2a2cszpmxcndoaye@localhost.localdomain>>) but no
one followed up on this so I just left the bibisect repos. I think they
were added by the bitergia folks; if there is consensus that they should
be removed I can do it, of course.

I think so - Xisco, can you confirm?

Actions

Copy link

#3

Updated by Xisco Fauli Tarazona over 6 years ago

Florian Effenberger wrote:

I mentioned that last August in the private marketing list (Message-ID:
<20170802100347.2a2cszpmxcndoaye@localhost.localdomain
<mailto:20170802100347.2a2cszpmxcndoaye@localhost.localdomain>>) but no
one followed up on this so I just left the bibisect repos. I think they
were added by the bitergia folks; if there is consensus that they should
be removed I can do it, of course.

I think so - Xisco, can you confirm?

I don't think I've read that email, anyway, I think it's good if we get rid of those bibisect repos in the dashboard

Actions

Copy link

#4

Updated by Guilhem Moulin over 6 years ago

Status changed from New to Feedback

2. In the last 2 years, 176 commits from László Németh are set as empty commits -> why?

That seems to reflect what can be found in the repository once we include all branches. The dashboard reports 256 commits between 2016-01-01 00:00:00 UTC and 2018-01-01 00:00:00 UTC, of which 171 are empty.

$ git log --oneline --since='2016-01-01 T00:00:00Z' --until='2018-01-01 T00:00:00Z' --branches --author='László Németh' | wc -l
258
$ git log --oneline --since='2016-01-01 T00:00:00Z' --until='2018-01-01 T00:00:00Z' --branches --author='László Németh' . | wc -l
87
$ echo $((258 - 87))
171

(I'm not aware of a better way to count empty commits, sorry :-P) Not sure where the 2 extra commits come from, perhaps wrong UTF8 encoding, or author vs. committer. Probably irrelevant anyway.

3. the number of commits by author in core is quite different comparing the dashboard with git-dm or git shortlog.

AFAIK the dashboard counts every single commit (probably excluding merges, didn't check) while git shortlog only counts those that are reachable from HEAD (or the optional revision argument).

In the dashboard, the top 5 from January 1st 2013, 19:00:00.000 to November 2nd 2017, 12:43:14.014 is
Caolán McNamara 20,075

$ git log --oneline --since='January 1st 2013, 19:00:00.000' --until='November 2nd 2017, 12:43:14.014' --branches --author='Caolán McNamara' . | wc -l
20184

Stephan Bergmann 13,119

$ git -C /srv/repositories/core.git log --oneline --since='January 1st 2013, 19:00:00.000' --until='November 2nd 2017, 12:43:14.014' --branches --author='Stephan Bergmann' . | wc -l
13142

Etc. Granted the counts are not exact but it's the same order of magnitude, and the missing handful of commits can be explained like above.

Do we want to count only the commits that are reachable from HEAD? Not sure how to that at the moment, I might have to ask the Bitergia folks.

For instance, Noel Grandin has 3 different affiliations in the Dashboard ( Link: http://goo.gl/nvJVAd ) because it counts all affiliation throughout the period of time, while git-dm only show the last affiliation.

And that's a feature not a bug, isn't it? I don't see why we should forget about devs' former affiliations and only pretend that all their commits are under their current affiliation.

Guilhem, is this something you can fix?

I don't see a problem yet. :-)

Actions

Copy link

#5

Updated by Guilhem Moulin over 6 years ago

Xisco Fauli Tarazona wrote:

Florian Effenberger wrote:

I mentioned that last August in the private marketing list (Message-ID:
<20170802100347.2a2cszpmxcndoaye@localhost.localdomain
<mailto:20170802100347.2a2cszpmxcndoaye@localhost.localdomain>>) but no
one followed up on this so I just left the bibisect repos. I think they
were added by the bitergia folks; if there is consensus that they should
be removed I can do it, of course.

I think so - Xisco, can you confirm?

I don't think I've read that email, anyway, I think it's good if we get rid of those bibisect repos in the dashboard

Alright, they're gone now. I couldn't find a way to do it in the dashboard but removing the ES indices did the trick:

curl -XDELETE -sS "http://127.0.0.1:9200/git_170123,git/items/_query" \
     -d '{"query":{"wildcard":{"repo_name":"git://gerrit.libreoffice.org/bibisect-*"}}}'

(Had to install the delete-by-query plugin, though.)

Actions

Copy link

#6

Updated by Guilhem Moulin over 6 years ago

Should we close this or try to make the dashboard only consider commits that are reachable from master (rather than any branch)?

Actions

Copy link

#7

Updated by Florian Effenberger over 6 years ago

Should we close this or try to make the dashboard only consider commits
that are reachable from master (rather than any branch)?

Can you ask Xisco about that? No hard feelings here

Actions

Copy link

#8

Updated by Florian Effenberger about 6 years ago

Florian Effenberger wrote:

Should we close this or try to make the dashboard only consider commits
that are reachable from master (rather than any branch)?

Can you ask Xisco about that? No hard feelings here

Ping? :)

Actions

Copy link

#9

Updated by Guilhem Moulin about 6 years ago

Florian Effenberger wrote:

Florian Effenberger wrote:

Should we close this or try to make the dashboard only consider commits
that are reachable from master (rather than any branch)?

Can you ask Xisco about that? No hard feelings here

Ping? :)

He's in the Watcher list :-P I also brought that up as a discussion item during the Jan 30 team call, and I don't recall hearing any objection either way (at least there is nothing in the minutes) so the ticket can probably be closed.

Actions

Copy link

#10

Updated by Xisco Fauli Tarazona about 6 years ago

Hello,
Sorry for my last response.
i don't have hard feelings about it either.
It's ok with me if it's closed.
Regards

Actions

Copy link

#11

Updated by Guilhem Moulin about 6 years ago

Status changed from Feedback to Closed

Xisco Fauli Tarazona wrote:

i don't have hard feelings about it either.
It's ok with me if it's closed.

OK, closing :-)

Actions

Copy link

#12

Updated by Guilhem Moulin over 5 years ago

FYI, since counting counting all commits (non only those in the master branch, but in any branch and/or reachable by any tag) caused some confusion, for the new dashboard I patched perceval to only count commits in the master branch. Cf. http://vm213.documentfoundation.org/app/kibana#/dashboard/Git .

Project

General

Profile

Infrastructure

Custom queries

Recently viewed projects

Task #2448

Dashboard stats wrong

Updated by Guilhem Moulin over 6 years ago

Updated by Florian Effenberger over 6 years ago

Updated by Xisco Fauli Tarazona over 6 years ago

Updated by Guilhem Moulin over 6 years ago

Updated by Guilhem Moulin over 6 years ago

Updated by Guilhem Moulin over 6 years ago

Updated by Florian Effenberger over 6 years ago

Updated by Florian Effenberger about 6 years ago

Updated by Guilhem Moulin about 6 years ago

Updated by Xisco Fauli Tarazona about 6 years ago

Updated by Guilhem Moulin about 6 years ago

Updated by Guilhem Moulin over 5 years ago