Internationalization: search and indexing

[eaquigley]

---

Author Name: Philip Durbin (@pdurbin)
Original Redmine Issue: 3741, https://redmine.hmdc.harvard.edu/issues/3741
Original Date: 2014-03-18
Original Assignee: Philip Durbin

---

Gustavo and I agreed today that we should think about how to internationalize search and indexing.

For example, the dataset field "title" is defined in one of the metadata tsv files. Titles for datasets are indexed under "title". Advanced search helps you populate "title:yourSearchTerm". But what would all of this look like in a different language. In a Spanish installation, would we index titles under the field "título"?

For now I'll assign this to Gustavo to think about when we'd like to put effort into this.

---

Related issue(s): #209
Redmine related issue(s): 3623

---

[posixeleni]

If we decide to add extra columns to the metadata tsv files for each language this may bloat the tables but there may be a way to figure this out in the db architecture so that it isnt an issue. I'm not a DB architect so will leave that to the experts.

[mercecrosas]

is there a better way to handle internationalization?

Mercè Crosas, Ph.D.
Director of Data Science, IQSS
Harvard University
http://iq.harvard.edu/merce-crosas

On Wed, Sep 10, 2014 at 10:57 AM, Eleni Castro [email protected]
wrote:

If we decide to add extra columns to the metadata tsv files for each
language this may bloat the tables but there may be a way to figure this
out in the db architecture so that it isnt an issue. I'm not a DB architect
so will leave that to the experts.

Reply to this email directly or view it on GitHub
https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_IQSS_dataverse_issues_326-23issuecomment-2D55127825&d=AAMCaQ&c=WO-RGvefibhHBZq3fL85hQ&r=MoES6dokjPLLcKaEAd7qaCuTcYZ4jLjEOBQnbbJ9BaA&m=Ir_YvgEwcmTyQJun8bir9-s_DPrXa6zh-d4vWrf0guU&s=0nzKRK1UgEIrmJKDuuWCmWfhJzOoa4EaLM3c6vDjcK8&e=
.

[pdurbin]

is there a better way to handle internationalization?

I'm not sure what the best way is. If you look at the internationalization efforts made by @Jian881219 in #783 for example, you'll see that changes are not being made to the database. Rather, properties files are written for each language (i.e. DatasetBundle_en.properties), which as far as I understand is the normal way to do it.

Since our metadata fields are data-driven (stored in the database), I can only assume that any translation effort would involve translating at least some parts of the TSV files that we load into the database. But as @posixeleni points out, this would be a lot of extra columns. Again, I'm not really sure what the approach should be.

[pdurbin]

Closing in favor of #209.