photog.social is one of the many independent Mastodon servers you can use to participate in the fediverse.
A place for your photos and banter. Photog first is our motto Please refer to the site rules before posting.

Administered by:

Server stats:

238
active users

#assessment

1 post1 participant0 posts today

YES!

"nformation systems that support the documentation and assessment of research activities should therefore, in future and as a common good, be open, freely usable and science-led"

The future development of assessment processes for research activity in the context of open science

doi.org/10.5281/zenodo.15836009

ZenodoThe future development of assessment processes for research activity in the context of open scienceDiscussion paper of the Task Force for Measure 4 of the Alliance 2021-2025 Open Access Strategy as part of the Alliance Initiative “Digital Information in Science”

Security through obscurity is still a policy in CA in 2025:

"CalMatters requested a copy of that document, but the Department of Technology declined to share it; Chief Information Security Officer Vitaliy Panych called doing so an unnecessary security risk."

"State claims there’s zero high-risk AI in California government—despite ample evidence to the contrary"

calmatters.org/economy/technol

#AI #risk #assessment #California

CalMatters · State claims there’s zero high-risk AI in California government—despite ample evidence to the contraryA state report provided to CalMatters says 200 agencies reported no automation around sensitive decisions. Some called the report befuddling.

A Comprehensive Framework For Evaluating The Quality Of Street View Imagery
--
doi.org/10.1016/j.jag.2022.103 <-- shared paper
--
“HIGHLIGHTS
• [They] propose the first comprehensive quality framework for street view imagery.
• Framework comprises 48 quality elements and may be applied to other image datasets.
• [They] implement partial evaluation for data in 9 cities, exposing varying quality.
• The implementation is released open-source and can be applied to other locations.
• [They] provide an overdue definition of street view imagery..."
#GIS #spatial #mapping #streetlevelimagery #Crowdsourcing #QualityAssessmentFramework #Heterogeneity #imagery #dataquality #metrics #QA #urban #cities #remotesensing #spatialanalysis #StreetView #Google #Mapillary #KartaView #commercial #crowsourced #opendata #consistency #standards #specifications #metadata #accuracy #precision #spatiotemporal #terrestrial #assessment

New study: #ChatGPT is not very good at predicting the #reproducibility of a research article from its methods section.
link.springer.com/article/10.1

PS: Five years ago, I asked this question on Twitter/X: "If a successful replication boosts the credibility a research article, then does a prediction of a successful replication, from an honest prediction market, do the same, even to a small degree?"
x.com/petersuber/status/125952

What if #LLMs eventually make these predictions better than prediction markets? Will research #assessment committees (notoriously inclined to resort to simplistic #metrics) start to rely on LLM replication or reproducibility predictions?

SpringerLinkChatGPT struggles to recognize reproducible science - Knowledge and Information SystemsThe quality of answers provided by ChatGPT matters with over 100 million users and approximately 1 billion monthly website visits. Large language models have the potential to drive scientific breakthroughs by processing vast amounts of information in seconds and learning from data at a scale and speed unattainable by humans, but recognizing reproducibility, a core aspect of high-quality science, remains a challenge. Our study investigates the effectiveness of ChatGPT (GPT $$-$$ - 3.5) in evaluating scientific reproducibility, a critical and underexplored topic, by analyzing the methods sections of 158 research articles. In our methodology, we asked ChatGPT, through a structured prompt, to predict the reproducibility of a scientific article based on the extracted text from its methods section. The findings of our study reveal significant limitations: Out of the assessed articles, only 18 (11.4%) were accurately classified, while 29 (18.4%) were misclassified, and 111 (70.3%) faced challenges in interpreting key methodological details that influence reproducibility. Future advancements should ensure consistent answers for similar or same prompts, improve reasoning for analyzing technical, jargon-heavy text, and enhance transparency in decision-making. Additionally, we suggest the development of a dedicated benchmark to systematically evaluate how well AI models can assess the reproducibility of scientific articles. This study highlights the continued need for human expertise and the risks of uncritical reliance on AI.

Excellent: "More than 100 institutions and funders worldwide have confirmed that research published in #eLife continues to be considered in hiring, promotion, and funding decisions, following the journal’s bold move to forgo its Journal Impact Factor."
elifesciences.org/for-the-pres

PS: This is not just a step to support eLife, but a step to break the stranglehold of bad metrics in research assessment. For the same reason, it's a step toward more honest and less simplistic assessment.

#Academia #Assessment #JIF #Metrics #Universities
@academicchatter

eLifeMore than 100 institutions and funders confirm recognition of eLife papers, signalling support for open scienceConversations with research organisations offer reassurance to researchers and highlight growing momentum behind fairer, more transparent models of scientific publishing and assessment.

So, the neuropsychologist I’ve been referred to to be assessed regarding ADHD, after two sessions, told me I score way too high on the autism questionnaire, so I must have autism and not ADHD (funny after my previous psychiatrist was sure about my ADHD, but not autism), and my ADHD questionnaire answers are less relevant because the problems “happen in social situations or special contexts” - and I was like ‘aren’t all the contexts special?’, but, of course, didn’t say it out loud - it seems, she’s just one of those who still consider autism and ADHD to be mutually exclusive.

And after that, she said she’s not a specialist in autism, only in ADHD, and I must look for another specialist to talk about autism assessment, and, of course, neither her health center nor my insurance company have one.

And people keep behaving as if those who don’t have an official diagnosis, don’t count! I mean, even financial side aside (hello 340 euros spent for nothing so far!), for someone with executive distinction to get through all those steps is insane and requires quite a lot of effort and dedication from anyone trying to - how can anyone expect all the ND people really go through it?

#autism
#ADHD
#assessment
#neurodivergent
@actuallyautistic

What will most transform #ScholComm in the next 10 years? A new survey of 90 #ECRs from 7 countries gives first place to #AI, followed closely by #OpenAccess and #OpenScience, followed by changes to #PeerReview.
onlinelibrary.wiley.com/doi/fu

While respondents thought AI would trigger more change than OA and OS, they were split on whether those changes would be good or bad. They were more united on the benefits of OA and OS.

I like this summary of the views of the Spanish respondents: "They believe that the much heralded new open and collaborative system is only possible if the evaluation of researchers changes and considers more than citations and includes altmetrics, publication in open platforms, repositories and so on."

#assessment : such valuation and an adjudging of the proper sum to be levied on the property

- German: die Beurteilung

- Italian: titolazione

- Portuguese: avaliação

- Spanish: evaluación

------------

Join our new subreddit for language learners @ reddit.com/r/LearnANewLanguage

reddit.comCrowdsourcing Languages • r/LearnANewLanguageWe are in the process of revitalizing this subreddit to help language learners. Thank you for being a part of our community!
Replied in thread

@veer66

Claims like "X language is for beginners" or "Y language isn't suitable for real problems" are just dressed-up versions of "X language sucks". They are pseudo-religious arguments, not technical ones. They are not useful in choosing technology to use.

A "real programmer" chooses languages, libraries, and other technical things based on utility, not holy wars (like "vi vs. Emacs", or "tabs vs. spaces").

e.g. For different problems and situations, you might choose a language because it is technically suited to a particular problem class. Or you might choose it because the group to work on the problem has deep experience with it, even though another language is slightly better suited to the problem. Or you might pick one based on a dozen other factors - and usually you will actually use more than one in making the choice.

Hollow assessments like "Python is for beginners" aren't useful. The people who make such statements are generally not particularly well-versed in the thing they are criticizing, and possibly not with programming/engineering in general.

If you want real assessments of the strengths and weaknesses of a language or other part of a tech stack, they're out there - but they will be articles and essays, not sentences.