Project

General

Profile

Task #3105

API Request From New Extensions Webpage

Added by Yusuf Keten over 1 year ago. Updated about 2 months ago.

Status:
In Progress
Priority:
Normal
Target version:
Team - Q3/2021
Start date:
Due date:
2021-08-08 (about 4 months late)
% Done:

70%


Description

I want to work on new Extension Manager feature through GSOC 2020. The idea is explained here: https://design.blog.documentfoundation.org/2016/11/11/additions-to-libreoffice/

But I need data to implement this idea. I think if the data I have mentioned below is sent from API in the new web page, it will be easier to implement for me.

I need:
Extension Name
Author
Description
License Type
Required Version
Comment Number
Comment Page Link
Positive and negative vote number
Download Number
Screenshot Link

If the above information can also be provided for Notebookbar, Fonts and Icons , I can also implement them.

Thanks.

#1

Updated by Christian Lohmaier over 1 year ago

copying remark from irc:

that is not really what you need for api within libreoffice - that is just getting detailed info for a single extension entry - so not really useful to present suggestions/results to the user.

Providing that info for a single extension in json or another parsable format is possible of course, but the important parts to fill a form from within LibreOffice is e.g. getting list of tags or searchterm or provide other filter criteria.
The current site already has a free-form-search along with a tag-filter, that later can be expanded by download-count and reviews (once the site has been in action for a while and had a chance to gather at least some amount of relevant numbers).
But especially reviews/comment count is tricky - as there is no incentive for users to rate an extension/you can only provide a rating after you downloaded and installed the extension. So you would have to go back to the extensionssite after having used the extension for a while to rate it.

Of course the scope of the extension discovery in LibreOffice can be extended to a call-to-action to rate installed extensions (but that needs to be tied to a configure switch that allows to disable this completely)

#2

Updated by Yusuf Keten over 1 year ago

You are right. If I at least get the search result and download link of extensions with JSON or something else, this will be great for me. I tried writing a program to get data from the extension website. However, searching and getting links took a lot of time depending on the number of results.

#3

Updated by Christian Lohmaier over 1 year ago

  • Status changed from New to Feedback
  • Assignee changed from Christian Lohmaier to Yusuf Keten

again: you're fully free to request any information in any form. If you need something to test, you can create some dummy output json as you wish to work with and I can then model the API in the extension site according to that template.
Don't halt your work from any "real" API, for your development you can just use JSON you create/tweak yourself as a first step. And once you have a format/set of data that works for you, I can start adding that to the site. (also incrementally, but I want some concrete endpoints (i.e. what URLs you want to call and what parameters you pass - for that silverstripe has the general url/actionname/id/param2 convention/syntax, but that's not a strict requirements https://docs.silverstripe.org/en/4/developer_guides/controllers/routing/ - so you can keep that in mind for your endpoints layout, but you're not tied to it)

Setting to Feedback (please change status back and reassign to me if you provided a sample json or similar that I can work of)

#4

Updated by Yusuf Keten over 1 year ago

  • Assignee changed from Yusuf Keten to Christian Lohmaier

I have prepared a document about my expectation from the API. There is an explanation of the API parameters. Also, there is a sample JSON file that consists of my expectation from the API. This document is my first work on an API request. Therefore, I am fully open to your feedback on this matter. Thank you.

https://docs.google.com/document/d/1T89OPoI36rbX6Wy139Bd-N0Z9yc3xOm8KTjCb8Ch7Cg/edit?usp=sharing

#5

Updated by Christian Lohmaier over 1 year ago

from IRC:
12:56 <cloph_away> some quick remarks to your api if viewBy is meant to specify sort, just call it sortBy :-) as for the response: an extension can have multiple downloads assigned and multiple releases
12:56 <cloph_away> For api use I guess one can just return the latest release, but still it could contain multiple donloads.
12:57 <yusufketen> Yes you are right, download links may be an array with version numbers and download links.
12:57 <yusufketen> About viewBy, you are again right :) I will change with sortBy.
12:57 <cloph_away> also the foobarNumber should be called foobarCount
12:58 <yusufketen> What is the foobar?
12:59 <yusufketen> download and comment numbers?
13:00 <cloph_away> If I understand it right, you will always pass OS and version, so from silverstripe's controller point of view it would be more natural to have that as part of the URL domain/api-base/valueForOS/valueForVersion/optionalSorting[?q=...&pageā€¦&per_page]
13:00 <cloph_away> but that's optional, using all as query parameters is fine as well.
13:01 <yusufketen> I agree with you about the URL.
13:02 <yusufketen> That will be better.
13:03 <cloph_away> so the only real problems is how to deal with multiple releases/multiple downloads per release.
13:04 <cloph_away> most natural would be to change the download URL to a array on its own just like tags.
13:05 <yusufketen> I think we just display the latest release and latest uploaded download link
13:05 <yusufketen> Because the UI will be basic to easily download and install extensions
13:05 <cloph_away> But still it would be hard to distinguish releases then. So either you would do additional queries to get the release notes/description for the release, or you also want to get the release description (and the releasenotes) as part of the response.
13:06 <yusufketen> If people want to download and install other versions, they can go webpage,download and install via extension manager.
13:10 <cloph_away> https://extensions.libreoffice.org/en/extensions/show/ancientgreek for example has two variants, in this case they are using a different release version, but there are also some that use different variants (different language for the UI element for example) and use the same release/version label.
13:10 <cloph_away> So only way to differenciate betwem then is by looking at the description and/or the release notes.
13:11 <cloph_away> (also the API requst sould include LO's UI language like the other requests do, to be able to provide a translated response if the entries are translated/to prefer entries with translated entries over English ones)

#6

Updated by Yusuf Keten over 1 year ago

I am aware that there is a problem with getting the latest release download link. So, is it possible to get the entire Release List(Version, name, description,license, and download link) with the API?

Also, I will send the system language as a parameter to API.

#7

Updated by Christian Lohmaier over 1 year ago

  • Category set to Extensions and Templates
  • Status changed from Feedback to In Progress
  • Target version set to Q3/2020

hasn't been updated after discussion on IRC, but I'll roll out a first version early next week, and then we can tune from there..

#8

Updated by Yusuf Keten over 1 year ago

Thank you so much. I'm grateful for your efforts.

#9

Updated by Yusuf Keten over 1 year ago

I added a new parameter to the API Document. (Tags) Also, I gave detailed explanations about the parameters and possible situations.

#10

Updated by Yusuf Keten over 1 year ago

We significantly reduced our expectations about the API. Can you review the document? https://docs.google.com/document/d/1T89OPoI36rbX6Wy139Bd-N0Z9yc3xOm8KTjCb8Ch7Cg/edit?usp=sharing

#11

Updated by Christian Lohmaier over 1 year ago

  • % Done changed from 0 to 70

added to live site with some changes - some discussed on irc already (but not reflected on the document), some technical prerequisites.

endpoint is https://extensions.libreoffice.org/api/v1/ (with v1 not being final yet, still subject to changes/tweaks as need arises) with either tags or extensions actions.

https://extensions.libreoffice.org/api/v1/tags/ will just return the list of tags along with a count of how many extensions have that tag assigned, and

https://extensions.libreoffice.org/api/v1/extensions/macOS/3.3?q=dictionary for example will return the extensions that are compatible with macOS and have a min-supported version of 3.3 and match the query string "dictionary" - I made os and version part of the URL since when running from LO those are always present.

other changes:
did change the labels of some of the returns from "fooNumber" to "fooCount" - e.g downloadCount (although that value amongst others still return null, also renamed some of the other fiels that either were redundant to match the internals of the site a little closer.
I did remove the commentURL from the response, as I don't see the point in having a URL for the comments section in an api repsonse

And for the time being the releases section doesn't include a download-URL yet - since that ties into the downloadCount - don't want to offer a direct download that's served by the webserver directly and thus bypasses any chance of bumping the download counter, so that requires some other changes on the site first.

One of the most important differences to the initial draft of the api is the pagination - the pagination offset is not per page, but by the result-item-index. So calling it "page" is misleading, although for the moment I left it at that, but would rather change that to "start" or "offset" or something along those lines.
The response contains a nextURL property that can be used to query the next results page. (for the moment I also didn't make the pagination limit configurable, right now it defaults to just three results - but that can be changed relatively easily

#12

Updated by Heiko Tietze over 1 year ago

Christian Lohmaier wrote:

https://extensions.libreoffice.org/api/v1/extensions/macOS/3.3?q=dictionary for example will return the extensions that are compatible with macOS and have a min-supported version of 3.3 and match the query string "dictionary" - I made os and version part of the URL since when running from LO those are always present.

This might lead to irritation when extension <foo_dictionary> is not tagged appropriately. Would be better to retrieve the extensions for a tag, whether via tag string or tag number.

#13

Updated by Christian Lohmaier over 1 year ago

as for the Tags api from your last comment - I guess we had a misunderstanding there - the tags API I mentioned was separate, and only give tags and extensioncount as described in the last comment, but not return the matching extensions along with that. To get extensions matching the tag, you do it as you'd do as a user with search and tag filters on the API, for example you would use

https://extensions.libreoffice.org/api/v1/extensions/?Tags[]=44 to get all extensions matching the tag with ID 44 (Writer) (can be combined with os/version and q/searchterm)

#14

Updated by Christian Lohmaier over 1 year ago

Heiko Tietze wrote:

Christian Lohmaier wrote:

https://extensions.libreoffice.org/api/v1/extensions/macOS/3.3?q=dictionary for example will return the extensions that are compatible with macOS and have a min-supported version of 3.3 and match the query string "dictionary" - I made os and version part of the URL since when running from LO those are always present.

This might lead to irritation when extension <foo_dictionary> is not tagged appropriately. Would be better to retrieve the extensions for a tag, whether via tag string or tag number.

That's for illustrative use only/what the API does/what the endpoint looks like, not for how to properly use the API from within LO :-) - I fully agree, to get dictionaries, you should use the dictionary tag as filter, and instead use your language's name as a query string.

#15

Updated by Heiko Tietze over 1 year ago

Christian Lohmaier wrote:

https://extensions.libreoffice.org/api/v1/extensions/?Tags[]=44 to get all extensions...

Perfect

#16

Updated by Christian Lohmaier about 1 year ago

  • Status changed from In Progress to Feedback
  • Assignee changed from Christian Lohmaier to Heiko Tietze

change status to feedback, since that is what it is - I need feedback to what in the API needs to change/whether it needs a change

#17

Updated by Yusuf Keten about 1 year ago

Can you check this JSON file? This completely works correctly. https://yusufketen.com/api/Impress.json

Because I couldn't validate the API JSON files from validation webpages like https://jsonformatter.curiousconcept.com/

As far as I see, there are HTML tags in JSON response which is come from the API. This is a problem for us.

Also, I'm not sure whether we delete non-ASCII characters(\u****) or not. I will ask this matter to my mentor.

#18

Updated by Florian Effenberger 12 months ago

  • Status changed from Feedback to In Progress
  • Assignee changed from Heiko Tietze to Christian Lohmaier
  • Target version changed from Q3/2020 to Q4/2020

Cloph, I guess I can re-assign the ticket to you for the moment?

#19

Updated by Florian Effenberger 12 months ago

  • Project changed from Infrastructure to Extensions
  • Category deleted (Extensions and Templates)
#20

Updated by Christian Lohmaier 10 months ago

  • Target version changed from Q4/2020 to Q1/2021
#21

Updated by Christian Lohmaier 10 months ago

  • Target version changed from Q1/2021 to Q2/2021

not enough info to properly do the api. It does what is in the initial design doc, and one of the requirements on the LO side was error resilience, to not fail if there is no api response or an unexpected response.
Fixing the mess needs more time, and likely requires me to study the LO code side for that.

#22

Updated by Christian Lohmaier 5 months ago

  • Due date set to 2021-07-25

plan to have it reworked for 7.2.0 rc2

#23

Updated by Florian Effenberger 5 months ago

  • Target version changed from Q2/2021 to Q3/2021

July 25 is Q3 -> rescheduling

#24

Updated by Christian Lohmaier 4 months ago

  • Due date changed from 2021-07-25 to 2021-08-08

might not be in time for rc2, but should be dynamic for final release using the scheme used by LO, changing LO's method is for later...

#25

Updated by Florian Effenberger about 2 months ago

Slipped through for a bit, should be done within the next days

Also available in: Atom PDF