x:HowmanypeoplevisitthepubliclibraryofNewYorkannually - PDF document

x:HowmanypeoplevisitthepubliclibraryofNewYorkannually l0:x:eq(x;count(y:people(y)^9e:visit(y;z:public(z)^library(z)^of(z;newyork);e)^annually(e))) y:x:library:public library system:annual visits(x;new york public library) a:13,554,002 x:WhatworksdidMozartdedicatetoJosephHaydn l0:x:works(x)^9e:dedicate(mozart;x;e)^to(haydn;e))) y:x:dedicated work(x)^9e:dedicated by(mozart;e)^dedication(x;e)^dedicated to(haydn;e))) a:fStringQuartetNo.19,HaydnQuartets,StringQuartetNo.16,StringQuartetNo.18,StringQuartetNo.17g Figure1:Examplesofsentencesx,domain-independentunderspeciedlogicalformsl0,fullyspeciedlogicalformsy,andanswersadrawnfromtheFreebasedomain.tationforthetargetdomain.Inourexample,pro-ducingeitherMR1,MR2oranothermoreappropri-ateoption,dependingontheQAdatabaseschema.Thistwostageapproachenablesparsingwithoutanydomain-dependentlexiconthatpairswordswithlogicalconstants.Instead,wordmeaningislledinon-the-ythroughontologymatching,enablingtheparsertoinferthemeaningofpreviouslyun-seenwordsandmoreeasilytransferacrossdomains.Figure1showsthedesiredoutputsfortwoexampleFreebasesentences.Therstparsingstageusesaprobabilisticcombi-natorycategorialgrammar(CCG)(Steedman,2000;ClarkandCurran,2007)tomapsentencestonew,underspeciedlogical-formmeaningrepresen-tationscontaininggenericlogicalconstantsthatarenottiedtoanyspecicontology.Thisapproachen-ablesustosharegrammarstructureacrossdomains,insteadofrepeatedlyre-learningdifferentgrammarsforeachtargetontology.Theontology-matchingstepconsidersalargenumberoftype-equivalentdomain-specicmeanings.Itenablesustoincorpo-rateanumberofcues,includingthetargetontologystructureandlexicalsimilaritybetweenthenamesofthedomain-independentanddependentconstants,toconstructthenallogicalforms.Duringlearning,weestimatealinearmodeloverderivationsthatincludealloftheCCGparsingde-cisionsandthechoicesforontologymatching.Fol-lowinganumberofrecentapproaches(Clarkeetal.,2010;Liangetal.,2011),wetreatallintermediatedecisionsaslatentandlearnfromdatacontainingonlyeasilygatheredquestionanswerpairs.Thisap-proachalignsnaturallywithourtwo-stageparsingsetup,wherethenallogicalexpressioncanbedi-rectlyusedtoprovideanswers.Wereportperformanceontwobenchmarkdatasets:GeoQuery(ZelleandMooney,1996)andFreebaseQA(FQ)(CaiandYates,2013a).Geo-Queryincludesageographydatabasewithasmallontologyandquestionswithrelativelycomplex,compositionalstructure.FQincludesquestionstoFreebase,alargecommunity-authoreddatabasethatspansmanysub-domains.Experimentsdemonstratestate-of-the-artperformanceinbothcases,includinganinepointimprovementinrecallfortheFQtest.2FormalOverviewTaskLetanontologyObeasetoflogicalcon-stantsandaknowledgebaseKbeacollectionoflogicalstatementsconstructedwithconstantsfromO.Forexample,KcouldbefactsinFreebase(Bol-lackeretal.,2008)andOwoulddenethesetofentitiesandrelationtypesusedtoencodethosefacts.Also,letybealogicalexpressionthatcanbeexecutedagainstKtoreturnananswera=EXEC(y;K).Figure1showsexamplequeriesandanswersforFreebase.Ourgoalistobuildafunctiony=PARSE(x;O)formappinganaturallanguagesentencextoadomain-dependentlogicalformy.ParsingWeuseatwo-stageapproachtodenethespaceofpossibleparsesGEN(x;O)(Section5).First,weuseaCCGandword-classinformationfromWiktionary1tobuilddomain-independentun-derspeciedlogicalforms,whichcloselymirrorthelinguisticstructureofthesentencebutdonotuseconstantsfromO.Forexample,inFigure1,l0de-notestheunderspeciedlogicalformspairedwitheachsentencex.Theparserthenmapsthisinterme-diaterepresentationtoalogicalformthatusescon-stantsfromO,suchastheyseeninFigure1. 1www.wiktionary.com libraryofnewyork NNnN=NPNPx:library(x)yfx:f(x)^loc(x;y)NYC �NnNf:x:f(x)^loc(x;NYC) Nx:library(x)^loc(x;NYC)Figure2:AsampleCCGparse.refertotheapplicationofaconstanttoasequenceofarguments.Weincludetypesforentitiese,truthval-uest,numbersi,eventsev,andhigher-orderfunc-tions,suchashe;tiandhhe;ti;ei.WeuseDavid-sonianeventsemantics(Davidson,1967)toexplic-itlyrepresenteventsusingevent-typedvariablesandconjunctivemodierstocapturethematicroles.CombinatoryCategorialGrammars(CCG)CCGsarealinguistically-motivatedformalismformodelingawiderangeoflanguagephenom-ena(Steedman,1996,2000).ACCGisdenedbyalexiconandasetofcombinators.ThelexiconcontainsentriesthatpairwordsorphraseswithCCGcategories.Forexample,thelexicalentrylibrary`N:x:library(x)inFigure2pairstheword`library'withtheCCGcategorythathassyntacticcategoryNandmeaningx:library(x).ACCGparsestartsfromassigninglexicalentriestowordsandphrases.ThesearethencombinedusingthesetofCCGcombinatorstobuildalogicalformthatcapturesthemeaningoftheentiresentence.Weusetheapplication,composition,andcoordinationcombinators.Figure2showsanexampleparse.5ParsingSentencestoMeaningsThefunctionGEN(x;O)denesthesetofpossiblederivationsforaninputsentencex.Eachderivationd=h;MibuildsalogicalformyusingconstantsfromtheontologyO.isaCCGparsetreethatmapsxtoanunderspeciedlogicalforml0.Misanontologicalmatchthatmapsl0ontothefullyspec-iedlogicalformy.Thissectiondescribes,withreferencetotheexampleinFigure3,theoperationsusedbyandM.5.1DomainIndependentParsingDomain-independentCCGparsetreesarebuiltusingapredenedsetof56underspeciedlexi-calcategories,49domain-independentlexicalitems,andthecombinatoryrulesintroducedinSection4.AnunderspeciedCCGlexicalcategoryhasasyntacticcategoryandalogicalformcontainingnoconstantsfromthedomainontologyO.Instead,thelogicalformincludesunderspeciedconstantsthataretypedplaceholderswhichwilllaterbereplacedduringontologymatching.Forexample,anounmightbeassignedthelexicalcategoryN:x:p(x),wherepisanunderspeciedhe;ti-typeconstant.Duringparsing,lexicalcategoriesarecreateddy-namically.WemanuallydeneasetofPOStagsforeachunderspeciedlexicalcategory,anduseWik-tionaryasatagdictionarytodenethepossiblePOStagsforwordsandphrases.Eachphraseisassignedeverymatchinglexicalcategory.Forexample,theword`visit'canbeeitheraverboranouninWik-tionary.Weaccordinglyassignitallunderspeciedcategoriesfortheclasses,including:N:x:p(x);SnNP=NP:xy9ev:p(y;x;ev)fornounsandtransitiveverbsrespectively.Wealsodenedomain-independentlexicalitemsforfunctionwordssuchas`what,'`when,'and`howmany,'`and,'and`is.'Theselexi-calitemspairawordwithalexicalcate-gorycontainingonlydomain-independentcon-stants.Forexample,howmany`S=(SnNP)=N:f:g:x:eq(x;count(y:f(y)^g(y)))containsthefunctioncountandthepredicateeq.Figure3ashowsthelexicalcategoriesandcombi-natorapplicationsusedtoconstructtheunderspeci-edlogicalforml0.Underspeciedconstantsinthisgurehavebeenlabeledwiththewordsthattheyareassociatedwithforreadability.5.2OntologicalMatchingThesecond,domainspecic,stepMmapstheun-derspeciedlogicalforml0ontothefullyspeciedlogicalformy.Themappingfromconstantsinl0toconstantsinyisnotone-to-one.Forexample,inFigure3,l0contains11constantsbutycontainsonly2.TheontologicalmatchisasequenceofmatchingoperationsM=ho1:::;onithatcantransformthe structureofthelogicalformorreplaceunderspeci-edconstantswithconstantsfromO.5.2.1StructureMatchingThreestructurematchingoperators,illustratedinFigure4,areusedtocollapseorexpandliteralsinl0.Collapsesmergeasubexpressionfroml0tocre-ateanewunderspeciedconstant,generatingalog-icalformwithfewerconstants.Expansionssplitasubexpressionfroml0togenerateanewlogicalformcontainingoneextraconstant.CollapsingOperatorsThecollapsingoperatordenedinFigure4amergesallconstantsinaliteraltogenerateasingleconstantofthesametype.Thisoperatorisusedtomapz:Public(z)^Library(z)^Of(z;NewYork)toPublicLibraryOfNewYorkinFigure3b.Itsoperationislimitedtoentitytypedexpressionsthatdonotcontainfreevariables.TheoperatorinFigure4b,incontrast,canbeusedtocollapsetheexpressioneq(x;count(y:People(y)^9e:Visit(y;PublicLibraryOfNewYork;e))^Annually(e))),whichcontainsfreevariablexontoanewexpressionCountPeopleVisitAnnually(x;PublicLibraryOfNewYork).ThisisonlypossiblewhenthetypeofthenewlycreatedconstantisallowedinOandthevariablexisfreeintheoutputexpression.SubsetsofconjunctscanbecollapsedusingtheoperatorinFigure4bbycreatingad-hocconjunctionsthatencapsulatethem.Disjunctionsaretreatedsimilarly.Performingcollapsesontheunderspeciedlogi-calformallowsnon-contiguousphrasestoberep-resentedinthecollapsedform.Inthisexam-ple,thelogicalformrepresentingthephrase`howmanypeoplevisit'hasbeenmergedwiththelogi-calformrepresentingthenon-adjacentadverb`an-nually.'Thisgeneratesanewunderspeciedcon-stantthatcanbemappedontotheFreebaserelationpublic library system annual visitsthatre-latestobothphrases.Thecollapsingoperationspreservesemantictype,ensuringthatalllogicalformsgeneratedbythederivationsequencearewelltyped.Thefullsetofallowedcollapsesofl0isgivenbythetransitiveclo-sureofthecollapsingoperations.Thesizeofthissetislimitedbythenumberofconstantsinl0,sinceeachcollapseremovesatleastoneconstant.Ateachstep,thenumberofpossiblecollapsesispolynomialinthenumberofconstantsinl0andexponentialinthearityofthemostcomplextypeinO.Fordo-mainsofinterestthisarityisunlikelytobehighandfortriplestoressuchasFreebaseitis2.ExpansionOperatorsThefullyspeciedlogicalformycancontainconstantsrelatingtomultiplewordsinx.Itcanalsousemultipleconstantstorep-resentthemeaningofasingleword.Forexample,Freebasedoesnotcontainarelationrepresentingtheconcept`daughter',insteadusingtworelationsrep-resenting`female'and`child'.Theexpansionoper-atorinFigure4callowsasinglepredicatetobesplitintoapairofconjoinedpredicatessharinganargu-mentvariable.Forexample,inFigure1,theconstantfor`dedicate'issplitintwotomatchitsrepresen-tationinFreebase.Underspeciedconstantsfroml0canbesplitonce.FortheexperimentsinSec-tion8,weconstraintheexpansionoperatortoworkoneventmodiersbuttheproceduregeneralizestoallpredicates.5.2.2ConstantMatchingTobuildanexecutablelogicalformy,allunder-speciedconstantsmustbereplacedwithconstantsfromO.Thisisdonethroughasequenceofcon-stantreplacementoperations,eachofwhichreplacesasingleunderspeciedconstantwithaconstantofthesametypefromO.TwoexamplereplacementsareshowninFigure3c.Theoutputfromthelastre-placementoperationisafullyspeciedlogicalform.6BuildingandScoringDerivationsThissectionintroducesadynamicprogramusedtoconstructderivationsandalinearscoringmodel.6.1BuildingDerivationsThespaceofderivationsistoolargetoexplicitlyenumerate.However,eachlogicalform(bothnalandinterim)canbeconstructedwithmanydiffer-entderivations,andweonlyneedtondthehighestscoringone.Thisallowsustodevelopasimpledy-namicprogramforourtwo-stagesemanticparser.WeuseaCKYstylechartparsertocalculatethek-bestlogicalformsoutputbyparsesofx.Wethenstoreeachinterimlogicalformgeneratedbyanop-eratorinMonceinahyper-graphchartstructure. iedconstantcuwithaconstantcOfromO.Theunderspeciedconstantcuisassociatedwiththese-quenceofwords~wuusedintheCCGlexicalen-triesthatintroduceditin.WeassumethateachoftheconstantscOinOisassociatedwithastringlabel~wO.Thisallowsustointroducevedomain-independentfeaturesthatmeasurethesimilarityof~wuand~wO.Thefeaturenp(cu;cO)signalsthereplacementofanentity-typedconstantcuwithentitycOthathaslabel~wu.ForthesecondexampleinFigure1thisfeatureindicatesthereplacementoftheunderspeci-edconstantassociatedwiththeword`mozart'withtheFreebaseentitymozart.Stemandsynonymyfeaturesstem(cu;cO)andsyn(cu;cO)signaltheexistenceofwordswu2~wuandwu2~wOthatshareastemorsynonymrespectively.StemsarecomputedwiththePorterstemmerandsynonymsareextractedfromWiktionary.AsingleFreebasespecicfeaturefp:stem(cu;cO)indicatesawordstemmatchbetweenwu2~wuandthewordllingthemostspecicpositionin~wuunderFreebase'shi-erarchicalnamingschema.Analfeaturegl(cu;cO)calculatestheoverlapbetweenWiktionarydenitionsfor~wuand~wO.Letgl(w)betheWiktionarydenitionforw.Then:gl(cu;cO)=Pwu2~wu;wO2~wO2
jgl(wO)\gl(wc)j j~wOj
j~wuj
jgl(wO)j+jgl(wc)jDomain-indepedentlexicalfeaturesallowthemodeltoreasonaboutthemeaningofunseenwords.Insmalldomains,however,themajorityofwordus-agesmaybecoveredbytrainingdata.WemakeuseofthisfactintheGeoQuerydomainwithfeaturesm(cu;cO)thatindicatethepairingof~wuwithcO.KnowledgeBaseFeaturesGuidedbytheobser-vationthatwegenerallywanttocreatequeriesywhichhaveanswersinknowledgebaseK,wede-nefeaturestosignalwhethereachoperationcouldbuildalogicalformywithananswerinK.Ifapredicate-argumentrelationinydoesnotexistinK,thentheexecutionofyagainstKwillnotreturnananswer.Twofeaturesindicatewhetherpredicate-argumentrelationsinyexistinK.direct(y;K)indicatespredicate-argumentapplica-tionsinythatexistsinK.Forexample,iftheappli-cationofdedicated bytomozartinFigure1ex-istsinFreebase,direct(y;K)willre.join(y;K)indicatesentitiesseparatedfromapredicatebyonejoininy,suchasmozartanddedicated toinFig-ure1,thatexistinthesamerelationshipinK.IftwopredicatesthatshareavariableinydonotshareanargumentinthatpositioninKthentheexecutionofyagainstKwillfail.Thepredicate-predicatepp(y;K)featureindicatespairsofpredicatesthatshareavariableinybutcan-notoccurinthisrelationshipinK.Forex-ample,sincethesubjectoftheFreebaseprop-ertydate of birthdoesnottakeargumentsoftypelocation,pp(y;K)willreifycon-tainsthelogicalformxy:date of birth(x;y)^location(x).Boththepredicate-argumentandpredicate-predicatefeaturesoperateonsubexpressionsofy.Wealsodenetheexecutionfeatures:emp(y;K)tosignalanemptyanswerforyinK;0(y;K)tosig-nalazero-valuedanswercreatedbycountingoveranemptyset;and1(y;K)tosignalaone-valuedanswercreatedbycountingoverasingletonset.Aswiththelexicalcues,weuseknowledgebasefeaturesassoftconstraintssinceitispossiblefornaturallanguagequeriestorefertoconceptsthatdonotexistinK.8ExperimentalSetupDataWeevaluateperformanceonthebenchmarkGeoQuerydataset(ZelleandMooney,1996),andanewlyintroducedFreebaseQuery(FQ)dataset(CaiandYates,2013a).FQcontains917questionsla-beledwithlogicalformmeaningrepresentationsforqueryingFreebase.Wegatheredquestionanswerla-belsbyexecutingthelogicalformsagainstFreebase,andmanuallycorrectinganyinconsistencies.Freebase(Bollackeretal.,2008)isalarge,col-laborativelyauthoreddatabasecontainingalmost40millionentitiesandtwobillionfacts,coveringmorethan100domains.WelterFreebasetocoverthedomainscontainedintheFQdatasetresultinginadatabasecontaining18millionentities,2072rela-tions,635types,135millionfactsand81domains,includingforexamplelm,sports,andbusiness.Weusethisschematodeneourtargetdomain,allow-ingforawidervarietyofqueriesthancouldbeen-codedwiththe635collapsedrelationspreviouslyusedtolabeltheFQdata. ParseFailures(20%) 1.Query inwhatyeardidmotorolahavethemostrevenue 2Query onhowmanyprojectswasjameswalkeradesignengineer StructuralMatchingFailure(30%) Query howmanychildrendoesjerryseinfeldhave 3.Labeled x:eq(x;count(y:people:person:children(jerry seinfeld;y))) Predicted x:eq(x;count(y:people:person:children(y;jerry seinfeld))) IncompleteDatabase(10%) Query howmanycountriesparticipatedinthe2006winterolympics 4.Labeled y:olympics:olympic games:number of countries(2006 winter olympics;y) Predicted y:eq(y;count(y:olympic participation country:olympics participated in(x;2006 winter olympics))) Query whatprogramminglanguageswereusedforaolinstantmessenger 5.Labeled y:computer:software:languages used(aol instant messenger;y) Predicted y:computer:software:languages used(aol instant messenger;y)^computer:programming language(y) LexicalAmbiguity(35%) Query whenwasthefridakahloexhibitatthephiladelphiaartmuseum Labeled y:9x:exhibition run:exhibition(x;frida kahlo)^ 6. exhibition venue:exhibitions at(philadelphia art museum;x)^exhibition run:opened on(x;y) Predicted y:9x:exhibition run:exhibition(x;frida kahlo)^ exhibition venue:exhibitions at(philadelphia art museum;x)^exhibition run:closed on(x;y) Figure9:Exampleerrorcases,withassociatedfrequencies,illustratingsystemoutputandgoldstandardreferences.5%ofthecasesweremiscellaneousorotherwisedifculttocategorize.guityintheFQdata,withoutsacricingtheabilitytounderstandingtherich,compositionalphenomenathatarecommonintheGeoQuerydata.QualitativeAnalysisWealsodidaqualitativeanalysisoferrorsintheFQdomain.Themodellearnstocorrectlyproducecomplexformsthatjoinmultiplerelations.However,thereareanumberofsystematicerrorcases,groupedintofourcategoriesasseeninFigure9.Therstandsecondexamplesshowparsefail-ures,wheretheunderspeciedCCGgrammardidnothavesufcientcoverage.Thethirdshowsafailedstructuralmatch,whereallofthecorrectlogi-calconstantsareselected,buttheargumentorderisreversedforoneoftheliterals.Thefourthandfthexamplesdemonstrateafailuresduetodatabasein-completeness.Inbothcases,thepredictedquerieswouldhavereturnedthesameanswersasthegold-truthonesifFreebasecontainedalloftherequiredfacts.Developingmodelsthatarerobusttodatabaseincompletenessisachallengingproblemforfuturework.Finally,thelastexampledemonstratesalex-icalambiguity,wherethesystemwasunabletode-termineifthequeryshouldincludetheopeningdateortheclosingdatefortheexhibit.10ConclusionWeconsideredtheproblemoflearningdomain-independentsemanticparsers,withapplicationtoQAagainstlargeknowledgebases.Weintroducedanewapproachforlearningatwo-stagesemanticparserthatenablesscalable,on-the-yontologicalmatching.Experimentsdemonstratedstate-of-the-artperformanceonbenchmarkdatasets,includingeffectivegeneralizationtopreviouslyunseenwords.Wewouldliketoinvestigatemorenuancedno-tionsofsemanticcorrectness,forexampletosupportmanyoftheessentiallyequivalentmeaningrepre-sentationswefoundintheerroranalysis.AlthoughwefocusedexclusivelyonQAapplications,thegen-eraltwo-stageanalysisapproachshouldallowforthereuseoflearnedgrammarsacrossanumberofdifferentdomains,includingroboticsordialogap-plications,wheredataismorechallengingtogather.11AcknowledgementsThisresearchwassupportedinpartbyDARPAun-dertheDEFTprogramthroughtheAFRL(FA8750-13-2-0019)andtheCSSG(N11AP20020),theARO(W911NF-12-1-0197),theNSF(IIS-1115966),andbyagiftfromGoogle.TheauthorsthankAnthonyFader,NicholasFitzGerald,AdrienneWang,DanielWeld,andtheanonymousreviewersfortheirhelpfulcommentsandfeedback.ReferencesAlshawi,H.(1992).Thecorelanguageengine.TheMITPress. Kwiatkowski,T.,Goldwater,S.,Zettlemoyer,L.,andSteedman,M.(2012).Aprobabilisticmodelofsyntacticandsemanticacquisitionfromchild-directedutterancesandtheirmeanings.Proceed-ingsoftheConferenceoftheEuropeanChapteroftheAssociationofComputationalLinguistics.Kwiatkowski,T.,Zettlemoyer,L.,Goldwater,S.,andSteedman,M.(2010).Inducingprobabilis-ticCCGgrammarsfromlogicalformwithhigher-orderunication.InProceedingsoftheConfer-enceonEmpiricalMethodsinNaturalLanguageProcessing.Kwiatkowski,T.,Zettlemoyer,L.,Goldwater,S.,andSteedman,M.(2011).LexicalgeneralizationinCCGgrammarinductionforsemanticparsing.InProceedingsoftheConferenceonEmpiricalMethodsinNaturalLanguageProcessing.Liang,P.,Jordan,M.,andKlein,D.(2011).Learn-ingdependency-basedcompositionalsemantics.InProceedingsoftheConferenceoftheAssoci-ationforComputationalLinguistics.Matuszek,C.,FitzGerald,N.,Zettlemoyer,L.,Bo,L.,andFox,D.(2012).Ajointmodeloflanguageandperceptionforgroundedattributelearning.InProceedingsoftheInternationalConferenceonMachineLearning.Muresan,S.(2011).Learningfordeeplanguageun-derstanding.InProceedingsoftheInternationalJointConferenceonArticialIntelligence.Steedman,M.(1996).SurfaceStructureandInter-pretation.TheMITPress.Steedman,M.(2000).TheSyntacticProcess.TheMITPress.Unger,C.,B¨uhmann,L.,Lehmann,J.,NgongaNgomo,A.,Gerber,D.,andCimiano,P.(2012).Template-basedquestionansweringoverRDFdata.InProceedingsoftheInternationalConferenceonWorldWideWeb.Wong,Y.andMooney,R.(2007).Learningsyn-chronousgrammarsforsemanticparsingwithlambdacalculus.InProceedingsoftheConfer-enceoftheAssociationforComputationalLin-guistics.Yahya,M.,Berberich,K.,Elbassuoni,S.,Ramanath,M.,Tresp,V.,andWeikum,G.(2012).Naturallanguagequestionsforthewebofdata.InPro-ceedingsoftheConferenceonEmpiricalMethodsinNaturalLanguageProcessing.Zelle,J.andMooney,R.(1996).Learningtoparsedatabasequeriesusinginductivelogicprogram-ming.InProceedingsoftheNationalConferenceonArticialIntelligence.Zettlemoyer,L.andCollins,M.(2005).Learningtomapsentencestologicalform:Structuredclas-sicationwithprobabilisticcategorialgrammars.InProceedingsoftheConferenceonUncertaintyinArticialIntelligence.Zettlemoyer,L.andCollins,M.(2007).OnlinelearningofrelaxedCCGgrammarsforparsingtologicalform.InProceedingsoftheJointConfer-enceonEmpiricalMethodsinNaturalLanguageProcessingandComputationalNaturalLanguageLearning.Zettlemoyer,L.andCollins,M.(2009).Learn-ingcontext-dependentmappingsfromsentencestologicalform.InProceedingsoftheJointCon-ferenceoftheAssociationforComputationalLin-guisticsandInternationalJointConferenceonNaturalLanguageProcessing.Zhang,C.,Hoffmann,R.,andWeld,D.S.(2012).Ontologicalsmoothingforrelationextractionwithminimalsupervision.InProceedsoftheConferenceonArticialIntelligence.