Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

in proquest, use new mag fields at level 0 #42

Open
f-hafner opened this issue Aug 26, 2023 · 0 comments
Open

in proquest, use new mag fields at level 0 #42

f-hafner opened this issue Aug 26, 2023 · 0 comments

Comments

@f-hafner
Copy link
Owner

currently, we're selecting based on pq_fields_mag which are imputed in load_proquest/correspond_fieldofstudy.py. Now that we have the predicted MAG fields, we could use this for selecting on field level 0.

This query computes a table similar to pq_fields_mag:

create table pq_fields_mag_future as 
select goid, ParentFieldOfStudyId AS FieldOfStudyId, sum(score) as score  
from (
    select goid, fieldofstudyid, ParentFieldOfStudyId, score 
    from pq_magfos 
    -- join parents at level 0
    inner join (
        select childfieldofstudyid, ParentFieldOfStudyId
        from crosswalk_fields
        where parentlevel = 0
    ) as crosswalk 
    on (pq_magfos.fieldofstudyid = crosswalk.childfieldofstudyid)
)
group by goid, parentfieldofstudyid

It could replace pq_fields_mag if we use the rank by score to replace the column position in the current table.
It would break some things: setup_linking may need to be adjusted, as well as link/topic_similarity_functions.py

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant